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ABSTRACT 


Questionnaires as social science tools are used increasingly to 
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researchers and managers in the design, construction, and use of mail 
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INTRODUCTION 


This bibliography was compiled in response to frequent requests for information 
on questionnaires by outdoor recreation researchers and the increasing use and misuse 
of questionnaires by investigators studying public reaction in the area of natural re- 
sources.— Questionnaires are a long standing tool for gathering data in social science 
research but are used increasingly by researchers and professional managers unfamiliar 
with the problems associated with their use in fields such as recreation and natural 
resource. The objective of this annotated bibliography is to make available a summary 
and appraisal of literature on the construction and use of questionnaires for research 
purposes. 


Several features are built into the bibliography to increase its usefulness: 


First, the annotations summarize the content and conclusions of the article 
rather than merely describing what the article is about. Of course, readers are fore- 
warned against quoting annotations directly without first consulting the original source 
because our interpretation of the original paper may be different from theirs. 


Second, accompanying most annotations is our subjective evaluation of the article. 
These evaluations may reflect different standards among the evaluators; nevertheless, 
they should be useful in helping readers decide which references warrant their further 
attention. 


Third, each citation contains a list of keywords geared specifically to topics 
covered in the article. These keywords are the basis for the index in the back of the 
bibliography to guide readers to those aspects of questionnaires in which they might 
be interested. 


In conclusion, readers contemplating the use of questionnaires are advised that 
the complexities of using questionnaires for research are often vastly underrated. The 
popularity of the method often rests on ignorance of associated problems of data 
analysis, bias, reliability, and validity of results. 


Persons desiring publications included in this bibliography should consult the 
reference source cited or write to the agency which sponsored the reference. Neither 
the Pacific Northwest Forest and Range Experiment Station nor the authors can provide 
copies of any references listed in this bibliography. 


1/ Portions of this work were conducted under cooperative agreement between 
the Forest Service and the University of Washington College of Forest Resources. 
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ANNOTATED BIBLIOGRAPHY 


Ash, Philip, and Edward Abramson 
1952. The effect of anonymity on attitude-questionnaire response. J. 
Abnorm. & Soc. Behav. 47(3): 722-723. 


Results from three attitude scales of ethnocentrism, political-economic 
conservatism, and anti-Negro prejudice administered to 96 college sociology 
students showed no significant difference between half the group required to 
sign answer Sheets and half who remained anonymous. 


EVALUATION: Extension of results should be made with caution--students are 
not necessarily representative of other sampling populations. 


KEYWORDS: Anonymous respondent. 


Azrin, N. H., W. Holz, and I. Goldiamond 
1961. Response bias in questionnaire reports. J. Consult. Psychol. 
25(4): 324-326. 


To investigate the validity of an earlier study of military personnel con- 
cerning behavioral reactions indicative of fear in combat, the authors used two 
questionnaire forms on two groups of college students totaling 160 people. A 
response bias was found toward certain symptoms of combat fear regardless of 
whether or not the symptom was alleged to occur. Correlations between stu- 
dents and fliers were high. Probably the same type of response bias influenced 
the combat fliers. Unless objective and direct means of measurement are 
available, the questionnaire responses may be independent of the behavior being 
studied. 


EVALUATION: Good paper reminding researchers of bias involved if certain 
questionnaire answers are expected and the respondent learns of this expectation. 


KEYWORDS: Bias: nonresponse, stated vs. actual behavior. 


Bachrack, Stanley D., and Harry M. Scoble 
1967. Mail questionnaire efficiency: controlled reduction of nonresponse. 
Publ. Opin. Quart. 31: 264-271. 


Contributors to the National Committee for an Effective Congress as of 
1960 were sampled with a 25-page questionnaire yielding a 63.5-percent re- 
sponse rate. A two-page followup yielded an additional 19.5 percent. These 
procedures were used to reduce nonresponse: a prequestionnaire explanatory 
letter, a prestamped and addressed return envelope, and a post card for re- 
questing report of findings. Followup letters repeated relevant information, 
and a second fo.lowup letter preceded the two-page questionnaire sent to all 
remaining nonrespondents. An identical form of this schedule was sent 1 month 
later, and the remaining nonrespondents were sent post cards containing 13 
short, census-type data questions. Three major reasons are suggested that 
account for nonresponse: infirmities of advancing age, personal disturbances 
in individual's life style (job change or bereavement), and hostility toward 


social survey research. Since a mailed questionnaire is an imposition, resis- 
tance is overcome by persuasion, persistence, attention to procedural techniques 
and details calculated to activate a potential respondent. 

EVALUATION: Good, practical suggestions for increasing returns along with 

a case study example to support conclusions. 


KEYWORDS: Followup: mail, post card 
Incentive: postage 
Cover letter. 


Bain, Read 

1931. Stability in questionnaire response. Am. J. Sociol. 37(3): 

445-453. 

A 61-item questionnaire given to 50 college freshmen was resubmitted 
to them 2-1/2 months later to measure the reliability of original results. One- 
fourth of the answers changed. Factual personal data changed less than factual 
family or subjective personal data. Girls changed their answers less often 
than boys. 
EVALUATION: The small sample and the author's occasional overenthusiastic 
and unsupported statements reduce the value of this article. Appears slanted 
to support the author's point of view. 


KEYWORDS: Reliability. 


Ball, Robert Jaudon 
1930. The correspondence method in follow-up studies of delinquent boys. 
J. Juvenile Res. 14: 107-113, illus. 


A brief questionnaire was distributed to parents to determine their satis- 
faction with the behavior of recently paroled sons. Frequency of meaningful 
response appeared correlated with child's IQ, but parents gave favorable 
report when opposite was known to be true. Inferences about parents' intelli- 
gence were drawn from the manner in which the questionnaire was completed. 
Rate of return was extremely low (40-percent return with only 45 percent 
properly filled out). 

EVALUATION: Dated, superficial, weak. It does alert researchers to the 
possibility of gaining information from the manner in which a questionnaire 
is completed. 


KEYWORDS: Bias: nonresponse. 


Barnette, W. Leslie, Jr. 
1951. The non-respondent problem in questionnaire research. J. Appl. 
Psychol. 34(6): 397-398. 


The solution to the nonresponse bias detection when a large nonrespondent 
group remains after two or three mail questionnaire followups is to randomly 
sample these nonrespondents and make persistent attempts to contact them. 

An example is given. 


EVALUATION: Good example of persistence paying off in reducing nonresponse 
bias. 


KEYWORDS: Bias: nonresponse 
Followup: telephone, mail. 


Bartholomew, Warren M. 
1963. Questionnaires in recreation Their preparation and use. 12 p. 
New York: Natl. Recreation Assoc, 


Results of a study by the National Recreation Association indicated that 
many questionnaires sent and received by recreation agencies are not effec- 
tively designed. Includes general comments on defining and clarifying the 
problem; types, content, wording, and sequence of questions; length of 
questionnaire; pretesting; distribution; ways to increase response; tabulation 
of data; and organization of the final reports. 


EVALUATION: Superficial instruction on use of questionnaires with no 
references to other literature. Perhaps useful to beginners but dangerous 
if they proceed with this reference alone. 
KEYWORDS: Questionnaire: design 

Applications. 


Barton, Allen H. 
1958. Asking the embarrassing question. Publ. Opin. Quart. 22: 
67-68, illus. 


A humorous, short article consisting solely of eight ways to ask the 
embarrassing question, "Did you kill your wife?" in a nonembarrassing way. 
For example, the casual approach, the direct approach, a sealed ballot box, 
a projective technique, etc. 


EVALUATION: Too brief with no discussion of important points. 


KEYWORDS: Questionnaire: design 
Question: wording. 


Bauer, Rainald K., and Frank Meissner 


1963. Structures of mail questionnaires: test of alternatives. Publ. 
Opin. Quart. 27: 306-311. 


A test was made to determine the validity of the following rule of thumb: 

"A series of logically interdependent questions generate more meaningful 
answers if put on one page of the questionnaire.'' Of the 198 complete answers 
received to a one-page questionnaire, 53.5 percent were absolutely correct, 
45.0 percent were conditionally correct, and 1.5 percent were absolute 
nonsense. Of the 765 complete answers received to a two-page question- 
naire, only 47.0 percent were absolutely correct, while the proportion of 
nonsense answers increased to 5.0 percent. Therefore, not only did the 
proportion of absolutely correct answers decline when two pages were used 


10. 


Mile 


12. 


but nonresponse and nonsense answers increased. Test results were, there- 
fore, consistent with the rule of thumb. 


EVALUATION: Insufficient information is given to allow the reader to check 
conclusions, but findings do support other literature. 


KEYWORDS: Questionnaire: design. 


Baur, E. Jackson 
1947, Response bias in a mail survey. Publ. Opin. Quart. 11: 
594-600, illus. 


The Veterans Administration mailed a questionnaire to 6,000 World War II 
veterans; 92 percent were returned. Characteristics of nonrespondents-- 
interest in questionnaire topic, formal education, age, marital status, and 
parenthood--were analyzed. Nonresponse bias was highest from difference 
in interest in questionnaire topic. Forty-nine percent of the nonrespondents 
had not attained a high school education, whereas of the respondents, 30 percent 
were without high school education. Those who were married or had little 
education were slower to respond. Age and parenthood did not influence res- 
ponse. Nonrespondents were not the same as late respondents. 


EVALUATION: Well-written, substantive article but a little weak in statistical 
analysis of results. 


KEYWORDS: Bias: nonresponse. 


Bender, Donald H. 
1957. Colored stationery in direct-mail advertising. J. Appl. Psychol. 
41(3): 161-164. 


Navy and Air Force officers and enlisted men totaling 12,056 received 
variously colored questionnaires to determine if color would increase the 
response rates. Response rates for various combinations of color including 
white, blue, green, canary, pink, and goldenrod ranged from 6.4 to 11.6 
percent. Mailings of colored questionnaires yielded higher returns as a 
group than white questionnaires, but the difference was statistically insig- 
nificant, 

EVALUATION: This experiment is one of few which tests color, but the 
return rate was so low that results are totally unreliable. 


KEYWORDS: Incentive: motivation 
Questionnaire: format. 


Bennett, E. M., R. L. Blomquist, and A. C. Goldstein 
1954. Response stability in limited-response questioning. Publ. Opin. 
Quart. 18: 218-223. 


Coefficients of stability were determined for 30 limited-response questions, 
for a 4-week, test-retest period, on 197 subjects. All stabilities were 


13. 


14, 


significantly greater than could be expected by chance. The questions were 
classified into four general categories, and the mean coefficients of stability 
for these classes of questions ranged from 0.713 to 0.970. Theoretical 
implications and polling procedures are described in detail. 


EVALUATION: Excellent discussion demonstrating problems of interpretation 
for fixed response—categories between informant and investigator. 
KEYWORDS: Validity 

Question: wording. 


Benson, Lawrence E. 
1946. Mail surveys can be valuable. Publ. Opin. Quart. 16: 234-241. 


Six advantages and seven disadvantages of the mail questionnaire for 
use in public opinion research are discussed based on Gallup Poll experience. 
The two main disadvantages are difficulty of obtaining returns from a 
representative cross section of the population and the proportionately greater 
number of returns that come from strongly biased respondents. Often 
results of surveys need to be adjusted to account for predicted changes in 
voter behavior. Mail questionnaires can be valuable provided their limita- 
tions are known and their results properly understood and correctly inter- 
preted. Actual survey problems are used as results, and extensive explana- 
tion is made of factors responsible for gross inaccuracies in the 1936 
Literary Digest presidential poll. 


EVALUATION: Excellent review of problems involved in the use of question- 
naires with population samples, specifically for political surveys. Good 
treatment of nonresponse. 


KEYWORDS: Advantages of questionnaires 
Bias: nonresponse 
Applications 
Disadvantages of questionnaires. 


Bevis, Joseph C, 
1948. Economical incentive used for mail questionnaire. Publ. Opin. 
Quart. 12: 492-493, 


During pretesting by Opinion Research Corporation, 10-, 25-, and 
50-cent war stamps were distributed with a mail questionnaire. The 25- 
and 50-cent stamps resulted in roughly the same return rate, but the 
10-cent stamp produced definitely lower returns. In the subsequent survey, 
only 25-cent war stamps were included. Of the 2,734 questionnaires, 84 
percent were returned, with one followup. A number of the latter respon- 
dents also returned the stamp. j 


EVALUATION: Information missing on size of pretest samples and percentage 
returns. Difficult to say that economic incentive or patriotism resulted 
in greater returns. 


15. 


16. 


Affe 


KEYWORDS: Pretest 
Incentive: money. 


Blankenship, Albert B. 
1940. The choice of words in poll questions. Sociol. & Soc. Res. 25 
(Sept. -Oct. ): 12-18. 


Individual words can elicit emotions and prejudices from respondents. 
Words may confuse or imply other than what the investigator intended. i 
If the investigator does use emotionally colored words, it should be clearly 
identified in his analysis just what was measured. Arrangement of words 
within phrases is important. Qualifying adjectives often provoke different 
responses to a question than when used without them. Author recommends 
a list of words to be avoided in questionnaires. Specific questions, clearly 
stated, are needed to avoid meaningless answers. 


EVALUATION: Dated but useful article on word choice. Good references to 
available literature with some excellent, specific examples. 
KEYWORDS: Question: wording. 


1943. Consumer and opinion research; the questionnaire technique. 
238 p. New York: Harper & Bros. Publ. 


This test includes topics on a historical perspective of questionnaires, 
the general technique of conducting a survey, formalizing the problem and 
study objectives, methods of collecting data, phrasing of questions, format 
of questionnaire, testing questionnaires for effectiveness, selecting sample, 
preparing and distributing materials, summarizing results, preparing the 
report, measuring the validity of surveys, and a case against the question- 
naire survey technique. 


EVALUATION: Text covers the range of problems encountered in question- 
naire surveys. It should be supplemented with recent findings. 


KEYWORDS: Textbook 
History 
Disadvantages of questionnaires. 


Boek, Walter E., and James H. Lade 
1963. A test of the usefulness of the post-card technique in a mail 
questionnaire study. Publ. Opin. Quart. 27: 303-306. 


The post-card technique, a procedure that maintains anonymity but 
still allows followups with nonrespondents, was used to determine if 178 
Health Department personnel would carry out their assigned functions in 
case of a nuclear attack. Replies were received from 86 percent of the 


18, 


19. 


20. 


sample; after a followup letter, total returns were 99 percent. Post cards 
were an important factor in this high percentage of return, although the 
type of people surveyed and their relationship with the organization had 
some effect on the return. 
EVALUATION: Rather superficial; response rate may have approached 99 
percent without the post card due to nature of topic and official positions 
of subjects. 
KEYWORDS: Anonymous respondent 

Followup: post-card, mail. 


Bradt, Kenneth 
1955. The usefulness of a post card technique in a mail questionnaire 
study. Publ. Opin. Quart. 19: 218-222. 


Questionnaires mailed to 5, 356 dropouts of an armed forces correspondence 


course included a post card to be returned separately to provide a means of 
followup without destroying anonymity of the respondent. Slightly more than 
4 percent of those who returned a questionnaire failed to return a post card. 
An 80-percent return was realized after one followup reminder. This post-card 
technique has broader application than is generally recognized. 
EVALUATION: Good case study of the use of post cards to preserve respondent 
anonymity. 
KEYWORDS: Anonymous respondent 

Followup: mail, 


Brennan, Robert D. 
1958. Trading stamps as an incentive in mail surveys. J. Marketing 
22(3): 306-307. 


From a census tract with characteristics approximating the average of 
the market area, 456 households were selected at random. In a Series of 
three experiments, half of the households were either sent or promised trading 
stamps or 25 cents, and half were given no incentives or had the length of their 
questionnaire reduced. The results of the three experiments indicate that small 
incentives do not increase the percent of mail returns significantly and that the 
additional cost does not warrant them. Such procedure merits consideration 
only if maximal information is desired. 
EVALUATION: Study supported by fact that return rates were extremely low-- 
22 to 29 percent, 


KEYWORDS: Incentive: money, reward. 


Britton, Joseph H., and Jean O. Britton 
1951. Factors in the return of adserionnalves mailed to older persons. 
J. Appl. Psychol. 35(1): 57-60. 


é 


21. 


22. 
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One study of 328 retired Y. M.C.A. secretaries received 68-percent returns 
from a preliminary questionnaire about retirement and 51. 9-percent returns from 
a longer one on the same subject. Followup letters to the 57 secretaries who had 
returned neither questionnaire resulted in a 50. 8-percent return. Generally the 
same procedure was followed for a second study of 2,853 retired teachers. A 
20-percent sample of nonrespondents was interviewed. Results show that the 
retired men in both studies had higher return rates. Former school administrators 
had higher returns than school teachers, and Y. M.C.A. secretaries with the 
highest returns were most apt to have served 25 years or more and to have moved 
more often since retirement. 


EVALUATION: Case study examples not tied to any broader framework with 
which to generalize results and conclusions. 


KEYWORDS: Questionnaire: length 
Bias: nonresponse 
Followup: mail. 


Brooks, Vernon 
1947, Can you trust mail questionnaires? Printers’ Ink 86, 88, 90, 92, 
96, 100. 


Registered voters in New York City were selected in a market survey to 
determine if the respondents were a representative cross section of society. Their 
results showed that the respondents tended to be better educated, and to be in the 
upper income and professional and managerial occupation classes. The 20- to 
29-year age group along with the 50-year-and-older group were underrepresented 
while the 30- to 49-year-old group was overrepresented. The authors conclude 
that if you are seeking opinions from a homogeneous group, mail questionnaires 
are fairly reliable, but if you seek opinions from a cross section of society, the 
mail survey is not reliable. 


EVALUATION: This article points to some areas that the researcher should 
consider. The results may be questioned due to biasing effect of a voter registra- 
tion sample being used as a representative cross section of people. 


KEYWORDS: Bias: representativeness. 


Brown, Morton L. 
1965. Use of a postcard query in mail surveys. Publ. Opin. Quart. 29: 
635-637. 


A sample of 523 physicians was subdivided into two subsamples. The first 
received a two-page query consisting of a cover letter, two screening questions, 
and space for patient diagnosis. The second subsample received a cover letter 
and a return post card containing only the two screening questions. For the 
original mailing, the post-card form elicited a 15-percent greater response, but 
this difference disappeared at the conclusion of the second mailing. Only 5-percent 
fewer contacts were required for the post-card query group. Most significantly, 
the feasibility of the post-card query is questionable because only 80 percent of 
the physicians responded to the additional query sent following receipt of the post- 
card form. The post-card query procedure was considered less efficient than the 
alternate query for eliciting the type of data needed. 


23. 


24, 


25. 


EVALUATION: That the initial post-card query may be less efficient than alterna- 
tive forms is in part dependent upon the circumstances in a particular study. The 
method should not be ruled out based on the conclusion drawn in this paper. 


KEYWORDS: Cover letter 
Questionnaire: format. 


Cahalan, Don 
1951. Effectiveness of a mail questionnaire technique in the army. Publ. 
Opin. Quart. 15: 575-578. 


A sample of 1,051 Army officers received a 23-page questionnaire dealing 
with Army interests. The cover letter stressed responsibility and requested a 
return within 5 days. Followup letters were not sent due to the 84-percent return 
rate. A large body of data available on each officer was used to check representa- 
tiveness of return. The respondents did not vary significantly from the nonrespon- 
dents. The questionnaire technique produced efficient and representative sampling 
in a single mailing and was more anonymous, less expensive, and faster than the 
interview technique. The questionnaire technique should work well on any other 
group with fairly strong lines of institutional control or traditions of responsibility. 


EVALUATION: The response rate might have differed if followup letters had been 
used. The report has little data but is worth reading in light of the use of "built-in" 
motivation for returning questionnaire. 


KEYWORDS: Mail questionnaire vs. interview 
Cover letter 
Incentive: motivation. 


Campbell, Donald T. 
1950. The indirect assessment of social attitudes. Psychol. Bull. 47(1): 
15-38. 


Covered are four types of attitude assessment tests: (1) nondisguised struc- 
tured: direct attitude test; (2) nondisguised-nonstructured: free response approach 
and essay; (3) disguised-nonstructured: "projective" techniques; and (4) disguised- 
structured: tests which approximate the objective testing of attitudes. There is 
no evidence that indirect tests obtain higher validity in measuring attitudes than 
do direct tests, but direct tests have much higher reliability than do indirect ones. 
Indirect tests have ability to pick up systematic, unconscious biases. 


EVALUATION: Comprehensive review of the literature of the 1940's dealing with 
indirect assessment of attitudes. 
KEYWORDS: Cover letter 

Questionnaire: format 

Validity 

Mail questionnaire vs. interview 

Attitudes. 


Cannell, Charles, and Floyd Fowler 


1963. Comparison of a self-enumerative procedure and a personal inter- 
view: a validity study. Publ. Opin. Quart. 27: 250-264. 


11 
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27. 


12 


Data of known validity from a sample of general hospitals were compared 
with data obtained from the responses of 462 persons in an interview procedure, 
and 465 persons in a self-enumerative procedure. Results show that when records 
are available or when reporting from respondents is done by proxy, the self enum- 
erative procedure is more accurate. In contrast, diagnosis and type of surgery 
are reported more accurately in an interview. This is consistent with the common 
observation that interviews about episodes involving threatening or embarrassing 
surgery is better than in self-enumerative procedures. Education of the respondent 
was more important in the interview. But respondent motivation was found more 
important in self-enumerative procedure, which stresses the need for developing 
special motivational techniques if this procedure is employed. 


EVALUATION: A well-written report tied directly to the data. Hypotheses are 
developed and tested, and conclusions consistent with the data are drawn. 


KEYWORDS: Mail questionnaire vs. interview 
Validity. 


Cantril, Hadley 
1940. Problems and techniques experiments in the wording of questions. 
Publ. Opin. Quart. 4: 330-332. 


In a national survey of 3,100 persons, alternate wording of two questions 
was given. The first question read as follows, ''Do you approve of Sumner Welles' 
visit to European capitals ?"' and its alternate read, ''Do you approve of President 
Roosevelt's sending Sumner Welles to visit European capitals?'' Response choices 
were: approve, disapprove, and no opinion. When Roosevelt's name was used, 
more people had opinions and although the percentage of people who approved the 
visit remained identical at 43 percent, more people disapproved when it was con- 
nected with Roosevelt. Response to the question without the President's name 
was 25 percent and response to the one with his name was 31 percent. This was 
more true of women than of men. 


EVALUATION: One should gain an appreciation for the wording of questions by 
reading this article. The point to be made is, be extremely cautious in wording 
questions. For example, in the first question the respondent is asked about 
Sumner Welles' visit, and in the second question, about Roosevelt's decision. 


KEYWORDS: Question: wording. 


Cavan, Ruth Shonle 
1933. The questionnaire in a sociological research project. Am. J. Sociol. 
38(5): 721-727. 


The reliability of an 80-item questionnaire was determined by retesting 123 
eighth-grade children after a 1-week interval. The replies of 62 pairs of siblings 
were compared, and 29 mothers were interviewed to check on factual questions. 
Factual questions about parents were completely reliable in 92 percent of the cases; 
facts about the home, in 95 percent of the cases; and those involving estimates of 
numbers, time, etc., 62 percent. There was 38-percent disagreement in attitudinal 
questions, but half were minor. Concludes that questions involving attitudes or 
estimates have lower reliability than factual questions. Reliability is increased 
by avoiding fine detail. 


28. 
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EVALUATION: Valuable even though dated. Poorly written with some lack in 
design and slight inconsistencies in reporting. 


KEYWORDS: Reliability 
Attitudes. 


Champion, Dean J., and Alan M. Sear 
1969. Questionnaire response rate: a methodological analysis. Soc. Forces 
47: 335-339. 


A total sample of 2,290 people, from the city directories of Knoxville, 
Nashville, and Chattanooga, were mailed questionnaires resulting in 42-percent 
return after one followup. Special delivery postage yielded returns of 61.2 per- 
cent versus 41.9 percent for regular postage. Six-page and nine-page question- 
naires were returned more often than three-page questionnaires. High and medium 
socioeconomic status (SES) persons responded more than low ones. Cover letters 
with an egoistic appeal (benefits the respondent would get from responding) resulted 
in greater response than letters with an altruistic appeal (benefits the research 
organization would get) from their response. Higher SES respondents favored 
altruistic-oriented cover letters, and lower SES respondents favored egoistic. 

16 references. 


EVALUATION: Informative study of response rate, clearly written with good 
literature review. 


KEYWORDS: Incentive: postage 
Cover letter. 


Clausen, John A., and Robert N. Ford 
1947. Controlling bias in mail questionnaires. Am. Statist. Assoc. J. 
42(240): 497-511. 


Nonresponse bias can be attacked on two fronts: (1) increase response and 
thus minimize unknown bias from nonrespondents and (2) estimate and correct for 
bias existing in nonresponding groups. Presents empirical data and literature 
review to spur returns. Concludes that a well-written cover letter plus followups | 
can motivate response. Airmail and special delivery letters often help. Person- 
alized greeting or signature do not. Interest in the questionnaire can be increased 
if questions of general or widespread interest are included. Estimating response 
bias may include consideration of respondent-nonrespondent homogeneity such as 
education or consistently different answers in successive waves of response after 
followups. 


EVALUATION: Review of bias problems plaguing questionnaires but some too 
specific to be of general help. 


KEYWORDS: Incentive: postage, motivation 
Followup: mail, personal contact 
Cover letter. 


Colley, R. H. 


1945. Don't look down your nose at mail questionnaires. Printers' Ink 
210(March): 21, 104, 106, 108. 
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31. 


32. 


33. 
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The mail questionnaire was once the principal tool in market research. Prior 
to World War II, however, the personal interview method became ingrained. War 
shortages of interviewers and transportation caused a reluctant return to the use 
of mail questionnaires. With careful planning, however, the questionnaire method 
is a valuable tool for consumer research. Problems of representativeness and 
respondent interest can be minimized if adequate safeguards are taken. 


EVALUATION: Inadequate, popularized discussion of problems in mail question- 
naires. Descriptions of procedures employed are incomplete. 


KEYWORDS: Mail questionnaire vs. interview 
History. 


Corey, Stephen M. 
1937. Signed versus unsigned attitude questionnaires. J. Educ. Psychol. 
28(2): 144-148. 


Study tested the belief that respondent-signed attitude tests are not as valid 
as those where respondent remains anonymous. An attitude scale containing 50 
statements reflecting sympathy or antipathy toward classroom cheating was admin- 
istered to 150 students, once with signatures required and once without. Both 
signed and unsigned tests had high split-half reliability coefficients with no signifi- 
cant differences between them. Correlation between scores on signed and unsigned 
tests was high (0.85 + 0.02). Mean scores reflected a slight, but statistically 
insignificant, tendency for more sympathetic attitudes toward cheating to be ex- 
pressed on anonymous papers, as did the total range in scores. The invalidating 
effects of signatures on questionnaires may be exaggerated. 


EVALUATION: Concise article reporting a well-executed study. 
KEYWORDS: Anonymous respondent, 


Crutchfield, Richard S., and Donald A. Gordon 
1947. Variations in respondents' interpretations of an opinion-poll question. 
Int. J. Opin. & Attitude Res. 1(3): 1-12. 


One Gallup Poll question was studied using 114 interviews to determine 
respondent interpretation and to determine how these affected the results of the 
questionnaire. The results varied according to the frame of reference used by 
the respondent. Men responded differently from women. Questionnaire results 
may be misleading if the frame of reference is not carefully analyzed. 


EVALUATION: Study clearly points to the need for careful question construction 
and cautious interpretation of the data in light of the respondent's interpretation 
of the questions. 


KEYWORDS: Question: wording 
Interpretation. 


Cuber, John F., and John B. Gerberich 
1946. A note on consistency in questionnaire responses. Am. Sociol. Rev. 
11(Feb. ): 13-15. 
Three questionnaires were filled out by 132 sociology students at 2-week 
intervals. Questions were of a factual, attitudinal, or evaluative nature. Questions 


34, 


35. 


were rearranged each time, and irrelevant questions inserted to disguise the pur- 
pose of the study. Seventy-two percent of the answers were consistent between 
applications. Factual questions showed the least reliability, differences in class 
levels or age of respondents were consistent. Questionnaire data should be 
accepted with caution until the methodological problems of reliability are investi- 
gated further. 


EVALUATION: A note of caution against blind acceptance of questionnaire data. 
Contradicts other findings that factual questions are most reliable. Does not 
suggest guidelines to improve reliability or validity. 


KEYWORDS: Reliability 
Validity. 


Davis, Robert A., and Edwin L. Barrow 
1935. <A critical study of the questionnaire in education. Educ. Admin. & 
Superv. 21(Feb.): 137-144. 


Over a period of 38 years, 500 questionnaires used in educational studies 
were reviewed to determine the rate of return, types of response called for, 
techniques used to increase returns, reliability, validity, and source of respondent 
information (fact, opinion, memory). Authors found that reports of many studies 
omit vital information such as length and number of returns of questionnaires. 
Reliability and validity of most questionnaires are probably so low that results 
are inconclusive. More appropriate statistical techniques are needed to determine 
reliability and validity. 

EVALUATION: Review article based on logical research criteria. Dated, but 
problems identified still exist. 
KEYWORDS: Bias: nonresponse. 


Deming, W. Edwards 
1944, Onerrors in surveys. Am. Sociol. Rev. 9(4): 359-369. 


Thirteen factors affecting the usefulness of surveys are: variability in 
response, differences between kinds and degrees of canvas, bias and variation 
arising from the interviewer, bias from sponsorship, imperfections in question- 
naire design and tabulation plans, changes taking place prior to availability of 
tabulations, bias from late reports, bias from unrepresentative selection of 
respondents or the period covered, sampling errors and biases, processing errors, 
and errors in interpretation. Errors from some of these factors are larger than 
commonly supposed; research must find ways of reducing errors and survey limi- 
tations. There is a special responsibility in presenting data to describe all diffi- 
culties to reduce errors of interpretation and utilization. As knowledge about 
bias and variability increases, their effects can be taken into account systemati- 
cally and surveys will be less hindered. 


EVALUATION: Excellent paper summarizing major sources of error and their 
impact on survey research results. 


KEYWORDS: Bias 
Questionnaire: design 
Interpretation. 
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Desing, Minerva F. 
1941. Suggestions to the novice in the mechanics of research. Sch. Rev. 
49(March): 206-212. 


A list of suggestions is given to assist students, clerical help, and novices 
in gathering, organizing, and treating data to guard against ambiguities, inconsis- 
tencies, and other irritations impeding progress (and validity) of a questionnaire 
study. Twelve suggestions are given for gathering data and 23 relevant to coding, 
analysis, and clerical procedure. 


EVALUATION: A good list of "how to do it" suggestions and precautions that 
would be vital to the novice and useful for the expert to review. 
KEYWORDS: Questionnaire: design 

Coding 

Analysis. 


Deutscher, Irwin 
1956. Physicians' reactions to a mailed questionnaire: A study in 
"resistentialism.'' Publ. Opin. Quart. 20: 599-604. 


A questionnaire headed ''Public Images of Female Occupations, '' was mailed 
to a random sample of 379 male physicians. Returns were not received from 43 
percent. The following social-psychological factors were isolated as potential 
reasons for nonresponse: resistance to demands to express stereotypes and broad 
generalizations; resistance to the limited alternatives presented in multiple-choice 
questions; resistance to value-laden, linguistic symbols such as "class"; and 
resistance to projective-type questions whose meaning and purpose are not clear 
to the potential respondent. The reduction of such forms of resistance would 
reduce nonresponse and incomplete returns. 
EVALUATION: Excellent suggestions worth consideration. 
KEYWORDS: Questionnaire: design 

Bias: nonresponse. 


1966. Words and deeds: social science and social policy. Soc. Probl. 
13(3): 235-254. 

Social science conclusions are often based on verbal response to questions 
about attitudes and behaviors. But as early as 1934, evidence by LaPiere showed 
there may be no positive relationship between what people say and do, and there 
may be a high inverse relationship. However, both empirical evidence and theo- 
retical rationale about this problem have been largely ignored. The social science 
model shows uncommon concern for methodological issues of reliability, neglect- 
ing problems of validity. A technology inappropriate to the understanding of human 
behavior has been developed. We do not know under what conditions a change in 
attitude anticipates a change in behavior and vice versa. Ideally, we should obtain 
information from the same population on verbal behavior and interaction on behavior 
under natural social conditions. The empirical evidence can best be summarized 
as reflecting wide variation in the relationships between attitudes and behaviors. 
Role theory can help us understand why man is constrained from acting as he 
"ought, '' and much can be learned from the undeveloped field of sociolinguistics. 


What do we know about the attitudinal consequences of behavioral change? To 
polarize attention upon two variables labeled "attitude" and "behavior" is to con- 
tinue down the wrong track. It is what goes on in between--the process--that 
needs attention. 


EVALUATION: Lengthy but relevant discussion of an important problem in the 
interpretation and meaning of questionnaire data. Evidence given demonstrates 
differences between "what people say" and ''what people do, "' 
KEYWORDS: Validity 

Interpretation 

Bias: stated vs. actual behavior. 


39. Diamond, Sigmund 
1963. Some early uses of the questionnaire: views on education and 
immigration. Publ. Opin. Quart. 27: 528-542. 


A selective historical review of the questionnaire. Earliest recording of 
the term in the English language is thought to be 1901. American dictionaries 
included it in the 1930's. Census, a form of the questionnaire research, is as old 
as government itself. In the 17th century, particularly in France, the question- 
naire was used for studies of social and political phenomena. Horace Mann used 
questionnaires to both gather and make opinion about public school systems. One 
early questionnaire study sought to ascertain from slave owners information about 
religious instruction provided Negro slaves. The questionnaire was used to gather 
reasons for immigration. Many questionnaires reveal factors of interest to the 
respondents rather than to the originator. 

EVALUATION: Interesting selection of historical examples but of limited appli- 
cability to questionnaires as a research tool. 


KEYWORDS: History. 


40. Dollard, John 
1948-49. Under what conditions do opinions predict behavior? Publ. Opin. 
Quart. 12: 623-632. 

Discussed are social and cultural factors which tend to sustain the validity 
of surveys and other factors which tend to make people's opinions unreliable for 
predicting their behavior. The following conditions may assist in understanding 
the relationship between opinion and action: (1) neurotics find it difficult to pre- 
dict their behavior when their own serious conflicts are involved; (2) persons with 
poor verbal skills may find it difficult to forecast their own behavior; (3) people 
who habitually go into effective action after thinking things over can best predict 
their own actions; (4) the test situation should not be corrupted by extraneous 
threats or rewards; (5) a man can best predict future situations if he has exper- 
ienced that situation before; (6) a man can predict a future situation, provided 
another experience does not change his mind first; and (7) a man can better pre- 
dict what he will do in a future dilemma if he is told exactly what the dilemma 
will be. 


EVALUATION: Excellent paper based upon psychological behavior theory that 
implicates the conditions of questionnaire validity. 
KEYWORDS: Validity 

Bias: stated vs. actual behavior. 
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42. 


43. 
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Donald, Marjorie 
1960. Implications of nonresponse for the interpretation of mail question- 
naire data. Publ. Opin. Quart. 24: 99-114. 


Extensive mail questionnaires to 2,768 dues-paying voluntary association 
members in 104 local units across the country yielded a 77. 3-percent return using 
three followups. At least half the nonrespondents did not actually support the 
organization. A significant relationship was found between amount of effort needed 
to get response and member involvement. The higher the involvement, in terms 
of active participation, knowledge and understanding of the organization, and 
loyalty to it, the fewer stimuli required to induce a response. Nonresponse to 
specific questions was similarly related. A phone call appeared to induce a cogni- 
tive reorientation of nonrespondents who were as involved as those responding to 
the preceding stimulus except their participation followed a more passive pattern. 
Nonresponse implications for data interpretation appear especially severe for 
attitudinal items and future behavior. Implications were less pervasive for esti- 
mates of respondent's own current behavior and insignificant for estimates of 
general group behavior of others. In the latter case, although there was more 
variability in response with decrease in involvement, there were no significant 
differences among the pooled responses at different levels of involvement. Costs 
can be minimized by selecting a cutoff point where inferences can be made about 
remaining nonrespondents. 

EVALUATION: "Significant" differences are reported in this paper. Questions 
that remain unanswered are how strong and important those differences are. 
Results in this paper are thus merely suggestive. 
KEYWORDS: Bias: nonresponse 

Incentive: motivation 

Followup: mail, telephone. 


Durant, Henry, and Irene Maas 
1956. Who doesn't answer? Brit. Psychol. Soc. Bull. 29: 33-34, 


Response is affected by the relationship between the subject of the inquiry 
and the recipient of the questionnaire. If it makes a difference who does not 
answer, then either the questionnaire technique should not be used or an attempt 
must be made to sample nonrespondents to see if they differ from respondents. 
In a test of university students, there were indications that people previously 
approached responded more readily a second time and that merely having to fill 
in two questions did not greatly increase cooperative response over having to fill 
in a 53-item questionnaire. 


EVALUATION: Good advice. Also suggests that questionnaire length is not a 
major factor in nonresponse. 


KEYWORDS: Questionnaire: length 
Bias: nonresponse. 


Eastman, R. O. 
1945. Dangers in direct-mail surveys. Printers' Ink 210(Jan.): 36, 40. 


Criticizes mail surveys and offers suggestions for improvement. Criticisms 
are: (1) mailed questionnaires automatically create two groups, the responsive 


| 


44, 


45. 


and the unresponsive and the resulting bias is difficult to determine; (2) data 
received are of insufficient depth to be useful for ascertaining anything other than 
strictly factual information. Suggestions are: (1) ascertain the distribution and 
characteristics of nonrespondents--it is not the number who respond that counts, 
but who responds; (2) don't cheat the data by slighting the analysis; few surveys 
are any good unless intelligently interpreted. Author concludes that the inexpensive 
characteristic of mail surveys is not an advantage, because one gets what he 

pays for. 


EVALUATION: Popular and homey in style, but several good points about 
problems inherent in mail surveys. 
KEYWORDS: Interpretation 

Disadvantages. 


Eckland, Bruce K. 
1965. Effects of prodding to increase mail-back returns. J. Appl. Psychol. 
49(3): 165-169. 


The less accessible subjects in a survey sample often exhibit unusual 
characteristics which may be of interest to the researcher. When telephone and 
certified mail techniques were used to obtain a 94-percent mail-back return from 
1,259 former college students, academic achievement variables were found to be 
related to the amount of prodding which was required to obtain a response. No 
significant relationship was found, however, between the amount of prodding and 
the veracity of the respondents' replies. Results indicate prodding may be war- 
ranted to determine effects of nonresponse. 


EVALUATION: Well-written, concise report describing one method to reduce 
nonresponse bias. 
KEYWORDS: Bias: nonresponse 

Followup: mail, telephone. 


Edgerton, Harold A., Steuart H. Britt, and Ralph D. Norman 
1947. Objective differences among various types of respondents to mailed 
questionnaire. Am. Sociol. Rev. 12(4): 435-444, illus. 


A followup study of all male contestants in the First Annual Science Talent 
Search using a mailed questionnaire revealed that: (1) 'Winner" contestants make 
almost 100-percent returns of the questionnaire for each of 3 followup years, (2) 
"Honorabie Mention" contestants make the next largest percentage of returns, and 
(3) "also ran" contestants have the lowest percentage of return. These results 
are consistent with previous studies which show that interest in the subject or 
sponsor influences returns. Analysis of Science Aptitude Examination scores of 
the "other" contestants for 4 followup years revealed that individuals replying 
faithfully to the questionnaire for 4 years tended to be superior on the examination. 
Their rising mean scores were directly related to frequency of replies, disregard- 
ing succession. Intensive and vigorous followup is a basic tenet in mail question- 
naire research, Otherwise the tendency will be to obtain replies from those who 
have a special interest in the subject under study, or who exhibit some character- 
istics different from the nonrespondents or from the casual or indifferent respon- 
dents. 
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46. 


47. 
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EVALUATION: Good example of a panel study on students. 


KEYWORDS: Bias: nonresponse 
Followup: mail. 


Ellis, Albert 
1946. The validity of personality questionnaires. Psychol. Bull. 43(5): 
385-440. 


This is a check of the validity of studies using questionnaires to distinguish 
neurotic from nonneurotic personalities. In each study reviewed, the results of 
personality assessment by questionnaire were compared with results obtained by 
other methods. Of 232 studies, 101 had correlation coefficients of r = 0.00-0. 39. 
Of an additional 42 studies specifically checking whether respondents overrated 
themselves, 36 showed overrating. Author concludes that group-administered 
personality questionnaires are able to discriminate one time out of two between 
groups of adjusted and maladjusted individuals. There is little indication they can 
be used at all to diagnose individual cases. Lengthy bibliography included. 


EVALUATION: An excellent and monumental review of validity of personality 
literature utilizing questionnaires. Bibliography, 360. 


KEYWORDS: Validity 
Literature review. 


1947. Questionnaire versus interview methods in the study of human love 
relationships. Am. Sociol. Rev. 12(Oct.): 541-553. 


Empirical study involving a questionnaire as a followup 1 year after a 
structured interview was given on love relationships of college women. Conclu- 
sions are: (1) for studying these types of behavior, the questionnaire method is 
at least as satisfactory as interviews; and (2) as questions become more ego-involv- 
ing, the questionnaire becomes better for producing relevant data. Does not rule 
out the interview as a method of providing depth or insight. Questionnaire is 
included. 


EVALUATION: Author does not generalize to other research areas, but informa- 
tion seems useful in many other behavior research efforts. 
KEYWORDS: Question: content 

Mail questionnaire vs. interview. 


1948. Questionnaire versus interview methods in the study of human love 
relations. II. Uncategorized responses. Am. Sociol. Rev. 13: 61-65. 


Part II of a study of 69 college girls interviewed on love and family relation- 
ships and surveyed 1 year later with an anonymous questionnaire to compare differ- 
ences in uncategorized (open-end) responses, revealed more self-revelatory or 
unfavorable responses to the anonymous questionnaire than to the interview. How- 
ever, probing during interviews increased such responses. 

EVALUATION: Mean scores reveal small differences. 
KEYWORDS: Mail questionnaire vs. interview 


Closed-end vs. open-end questions 
Anonymous respondent. 


49, 


50. 


51. 


Ellis, Robert A., Calvin M. Endo, and J. Michael Armer 
1970. The use of potential nonrespondents for studying nonresponse bias. 
Pac. Sociol. Rev. 13(2): 103-109. 


Data obtained through study of 412 college students. Followup efforts of 
two questionnaires left 19 nonrespondents. Data indicate that late respondents do 
not provide a suitable basis for estimating the characteristics of nonrespondents, 
Attempts to distinguish between potential nonrespondents and late respondents 
indicate only the direction of nonresponse bias, not its magnitude. The researcher's 
main recourse is to extend the scope of the analysis beyond the self-report data 
and to gain information on subject characteristics from outside data sources. 
Although minor effects of nonresponse are evidenced on social background variables 
in this and other studies, it is possible that serious nonresponse error may be 
present among other variables. 
EVALUATION: Excellent paper on nonresponse bias with 35 references for support. 
KEYWORDS: Bias: nonresponse 

Early vs. late response. 


Erdos, Paul L. 

1957a. How to get higher returns from your mail surveys. Printers' Ink 

258(Feb. ): 30-31. : 

The following 8 factors are important to securing a high percentage of returns: 
prestige of organization doing the survey, interest of respondents, readability and 
length of questionnaire, wording of letter and questionnaire to convince respondent 
of his importance to the survey, list of benefits to the recipient, assurance of 
respondent anonymity, offer of a report of results, stamped envelope for reply 
(airmail preferable). 

EVALUATION: Excellent rules of thumb but lacking in criteria for evaluating 
effects of treatment. 


KEYWORDS: Questionnaire format 
Cover letter 
Anonymous respondent 
Incentive: motivation, postage. 


1957b. Successful mail surveys: high returns and how to get them. 
Printers' Ink 258(March): 56-60, illus. 


If questionnaire return rates are low, then further mail survey techniques 
should include: post-card reminders, several mailings, telegrams, obligating 
respondent to participate in advance of receiving the questionnaire, or a premium. 
Premiums should attract the respondent, introduce no bias, and should be inex- 
pensive and easy to mail. Three case histories are discussed. Premiums as a 
moneysaving device and the possibility of premium bias are discussed. Testing 
premium bias revealed a higher average income for premium-respondents than 
for nonpremium respondents. 


EVALUATION: Excellent paper with good rules of thumb. 


KEYWORDS: Incentive: reward 
Followup: post-card, mail. 
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52. 


53. 


54. 
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Ferber, Robert 


1948a. The problem of bias in mail returns: a solution. Publ. Opin. Quart. 
12: 669-676. 


Returns are analyzed to determine, without using followups, whether a set 
of mail returns is, or is not, representative. Returns are grouped on the basis 
of their arrival time. If the distribution of item responses of the late respondents 
differs markedly from that of the early respondents, bias is presumed to exist, 
and followup interviews are necessary. Random order tests are not foolproof 
indicators of mail bias. Error may be inherent in the theory of probability on 
which these tests are based, and bias may be independent of the order of return 
due to certain nonmeasurable characteristics of the respondents. However, in 
most cases where bias is reflected in measurable characteristics, these tests do 
provide an objective means of testing for bias in mail surveys. Application of 
these bias tests to the pretest of a large scale survey would also be advantageous. 


EVALUATION: Clues as to the representativeness of an incomplete sample may 
be obtained by assuming that late respondents resemble nonrespondents. This 
assumption cannot, however, be substantiated, and one is left to proceed on faith. 
A worthwhile paper--should be read. Also see Robert Ferber, 1950, Publ. Opin. 
Quart. 14: 193-197, and Publ. Opin. Quart. 13: 562-563, for further discussion 
of this paper in a letter to the editor. Also see Ford and Zeisel (1949). 
KEYWORDS: Bias: nonresponse, representativeness 

Early vs. late response. 


1948b. Which--mail questionnaires or personal interviews? Printers' Ink 
222(7): 44-47, 61, 64-65, illus. 


Advantages of the mail questionnaire compared with the personal interview 
include the following: a wider distribution of the sample, a smaller staff, lower 
cost, opportunity for frank responses, more ease in obtaining opinions of all 
family members, and easier access to high-income groups. Disadvantages 
include: nonresponse bias, low response rates from persons with low income 
and education, necessity of followups, little knowledge of causes and reasons for 
respondent's actions and attitudes, no assurance that the correct person responded, 
and length of time until replies are received. Each item is fully discussed, and 
the general conclusion is that neither the questionnaire nor the interview technique 
is superior. Conditions of the problem define the relative desirability of the two 
techniques. 


EVALUATION: Well-balanced conclusions, good summary of questionnaire and 
interview use. 
KEYWORDS: Mail questionnaire vs. interview 

Advantages of questionnaires 

Disadvantages of questionnaires. 


1966. Item nonresponse in a consumer survey. Publ. Opin. Quart. 30: 
399-415. 


55. 


56. 


A study of 14, 600 usable questionnaires out of 40,000 mailed to subscribers 
of Consumers Unton analyzed item nonresponse by characteristics of respondents 
and by type, position, and repetition of questions. One-third of the questionnaires 
contained two or more unanswered items, and one out of seven contained six or 
more items unanswered. Item nonresponse was higher for females, older people, 
and those in households where the main wage earner had only a grade school edu- 
cation or was retired or a housewife. Frequency of item nonresponse increased 
for questions requiring some thought or effort by respondents, but the pattern of 
item nonresponse by respondent characteristics was very similar for different 
types of questions. No relationship was found between frequency of item nonres- 
ponse and the position of the question on the questionnaire. Although the effect of 
item nonresponse on parameter estimates is not easily determined, item nonres- 
ponse can reduce substantially the reliability of estimates. In particular, if 
allowance is made for possible bias due to this source in estimates of the variances 
of parameters, the size of the standard errors may turn out to be substantially 
higher than without such adjustment. 


EVALUATION: Validity of findings is questionable due to low rate of return. 
KEYWORDS: Bias: nonresponse. 


Ferriss, Abbott L. ; 
1951. A note on stimulating response to questionnaires. Am. Sociol. Rev. 
16(2): 247-249, 


The effectiveness of three techniques for stimulating response to mailed 
questionnaires was tested in introductory sociology courses in the 11 Southeastern 
States. The use of a ''deadline"' stimulated an immediate, heavy response. By 
providing stamped addressed envelopes in the first and second waves of question- 
naires, the flow of returns was increased. Postal cards as reminders were com- 
pletely ineffective. Costs per return were slightly higher for the segment of the 
population which was not provided with stamped, return envelopes and which was 
not prodded. Responses were most frequently mailed on Thursday and Friday; 
least frequently, on Saturday and Sunday. 

EVALUATION: Data support conclusions. 
KEYWORDS: Incentive: motivation, postage 
Followup: post-card. 


1963. Applications of recreation surveys. Publ. Opin. Quart. 27: 443-454, 


Three types of recreation surveys and their use are discussed: travel studies 
from persons passing a transportation point; onsite studies from persons engaging 
in an activity under study; and household surveys from persons at home. Variables 
usually studied include: demography, trip characteristics (mode of travel, etc.), 
time andcost of trip, attitude and motivation of traveler, and decisionmaking factors. 
Several studies are summarized and their applications are discussed. Author 
suggests that more attitude, motivation, and decisionmaking factors be included 
to help explain additional variance. 


EVALUATION: Although a moderately well-constructed typology of existing recrea- 
tion surveys, includes nothing specific to the questionnaire method. 
KEYWORDS: Applications. 
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58. 


59. 


60. 
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Fischer, Robert P. 
1946. Signed versus unsigned personal questionnaires. J. Appl. Psychol. 
30: 220-225. 


A psychological problem checklist was given to 102 female psychology stu- 
dents, first with and then without signatures being required. The interim between 
testings was 1 week. The results indicated that the mean number of problems 
listed as not serious did not vary significantly under the two conditions of adminis- 
tration, but that the mean number of serious problems listed tended to be signifi- 
cantly greater when signatures were not required. It would appear that the use of 
signatures on personal questionnaires might have an inhibitory effect on the honesty 
and frankness of the people responding to them. 

EVALUATION: Good example of value of anonymity, especially for sensitive 
subjects under investigation. 
KEYWORDS: Anonymous respondent. 


Ford, Neil M. 
1968. Questionnaire appearance and response rates in mail surveys. J. 
Adver. Res. 8(3): 43-45. 


This mail survey to consumer shoppers tested printed cover letters against 
mimeographed cover letters to see if the printed ones would bring in a greater 
return than the mimeographed letters. Questionnaire and cover letters were sent 
to 1,556 consumers. The printed letter had a return of 22 percent, and the mimeo- 
graphed letter had a return of 20 percent. Chi-square statistic showed no signifi- 
cant difference between them. 


EVALUATION: The extremely low response rate would have made the results 
unreliable even if there had been a significant difference in returns between the 
two cover letters. This survey adds further evidence that no one variable is 
responsible for high return rates, but a combination of many is necessary. 
KEYWORDS: Cover letter. 


1969. Consistency of responses in a mail survey. J. Adver. Res. 9(4): 
31-33. 


Questionnaires were mailed to 1,556 households. Of the 319 responding to 
the questionnaire on consumer shopping behavior, 159 were selected for interview 
to determine how consistently questions could be answered. Questions dealing 
with facts such as socio-economic factors elicited the highest degree of consistency. 
Attitudinal or opinion questions had the lowest average consistency between answers. 
EVALUATION: One should gain an appreciation for the difficulty of using attitudinal 
and opinion questions from reading this paper. 
KEYWORDS: Question: type, content 

Reliability. 


Ford, Robert N., and Hans Zeisel 
1949, Bias in mail surveys cannot be controlled by one mailing. Publ. 
Opin. Quart. 13: 494-501. 


61. 


62. 


Experimental data fail to show that a test for random order of response can 
detect bias after one mailing, as has been suggested by Ferber (1948a). The con- 
clusion is that Ferber's statistical test is not sensitive enough to assure the analyst 
that a "no bias" finding indicates that no bias actually exists. The theory that late 
respondents to a single mailing are similar to the nonrespondents appears to have 
been offered by Ferber without support. The proposed procedure of analysis is 
dangerous in that it almost inevitably leads to a "no bias" verdict when bias can be 
shown to exist by the more difficult and costly techniques it seeks to avoid. 
EVALUATION: Criticizes Ferber's 1948 paper: ''The problem of bias in mail 
returns: a solution.'' Worth reading in conjunction with Ferber's paper. 
KEYWORDS: Bias: nonresponse 

Early vs. late response. 


Frank, Benjamin 
1935. Stability of questionnaire response. J. Abnorm. & Soc. Psychol. 
30: 320-324, 


The Bernreuter Personality Inventory was personally administered to 76 
New Jersey prison inmates. Reliability coefficients (about 0.80) were smaller 
than reported by Bernreuter. Neutral items showed the highest index of change 
(average, 32.5 percent) with positive (yes) items and negative (no) items averaging 
about 21-percent change. Concludes that until the influence of situation and factors 
motivating response can be definitely stated, the personality questionnaire will 
remain a dubious clinical instrument. 
EVALUATION: Dated study, with confusing presentation. 


KEYWORDS: Reliability. 


Franzen, Raymond, and Paul F. Lazarsfeld 
1945. Mail questionnaire as a research problem. J. Psychol. 20: 293-320. 


Although the main focus of this paper is on a study of who returns mail 
questionnaires, the authors also review several advantages and disadvantages of 
mail questionnaire use. Bias resulting from self-selection may or may not be 
significant and must be evaluated with regard to the topic of interest. The study 
reported in this paper compared personal interview replies from those who did 
respond with those who did not respond to a mail questionnaire to determine (1) the 
nature of any nonresponse bias which may occur, and (2) the bias which results 
from the two different methods of inquiry. It was found that consistent and statisti- 
cally significant differences between respondents and nonrespondents were present. 
However, for practical purposes, the differences were either not large or could be 
corrected by weighting. Mail questionnaires have an advantage over interviews. 
For cultural subjects, they are more informative and show a freer admission of 
unusual activities or interests. On questions involving buying power, the mail 
response shows less reluctance to divulge information. An appendix is included 
in which interviewer bias (selection bias and opinion bias) is discussed. 


EVALUATION: This paper takes a comprehensive look at several important prob- 
lems associated with mail questionnaires and personal interviews. The study 
discussed is a good example of how problems of bias can be evaluated. 
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KEYWORDS: Bias: nonresponse 
Mail questionnaire vs. interview 
Reliability 
Advantages of questionnaires 
Disadvantages of questionnaires. 


Frazier, George, and Kermit Bird 
1958. Increasing the response of a mail questionnaire. J. Marketing 
23(2): 186-187. 


Two Idaho counties were chosen to test the effectiveness of adding a hand- 
written postscript which read, 'P.S. We need your help in this report. Could 
you please send it in promptly?'"' There were 777 questionnaires sent out with the 
postscript included and 781 sent without the postscript. A chi-square test showed 
a 6. 7-percent difference between them with 31.4 percent for letters with the post- 
script and 24. 7-percent return for letters without. 

EVALUATION: Poorly written paper, with some inaccurate reporting of findings. 
Data point to desirability of personalizing a questionnaire as much as possible. 
KEYWORDS: Cover letter 

Incentive: motivation. 


Freeman, Linton C., and Tiirkoz Ataov 
1960. Invalidity of indirect and direct measures of attitude toward cheating. 
J. Personality 28(4): 443-447. 


The study was to determine the relationship between overt behavior and a 
direct question and three types of indirect attitude items to predict the behavior. 
Thirty-eight university freshmen and sophomores were ranked in terms of 
observed classroom cheating and polled with both direct and indirect items about 
cheating. All correlations were insignificant, and neither direct nor indirect 
questionnaire items were of any uSe in predicting behavior. These results thus 
cast doubt upon the validity of direct or indirect questions for the assessment of 
certain types of overt behavior. 


EVALUATION: Good, interesting study demonstrating a major shortcoming of 
questionnaire research. 


KEYWORDS: Validity 
Disadvantages of questionnaires 
Bias: stated vs. actual behavior. 


Gannon, Martin J., Joseph C. Nothern, and Stephen J. Carroll, Jr. 
1971. Characteristics of nonrespondents among workers. J. Appl. 
Psychol. 55(6): 586-588, illus. 


Questionnaires were sent to 552 checkers in 14 food stores to identify their 
attitudes about supervisory practices, the social climate, and personnel policies 
in the firm. After a return of 63 percent, nonrespondents were compared with 
respondents on the basis of education, length of employment, marital status, sex, 
age, number of children, and supervisory rankings. A chi-squared analysis 
showed that nonrespondents were typically lower in education, of single status, 


male, younger or older than the 30- to 49-year-old age group, and ranking in the 
bottom fifth of all employees. Length of employment and number of children were 
not found correlated with nonresponse. 
EVALUATION: Brief paper presenting findings only. Discussion and implications 
of findings are lacking. 

KEYWORDS: Bias: nonresponse. 


66. Gee, Wilson 
1950. The survey method, p. 300-329. In Social science research 
methods. New York: Appleton-Century-Crofts. 


The mail questionnaire is introduced as a supplement to a more elaborate 
survey method. The chief advantages are its range, economy, and usefulness in 
social research for: supplementing a study, providing preliminary information 
necessary for a project outline, obtaining primary data, enlarging and supple- 
menting the background of a problem, checking for changes, checking secondary 
data already available, and disclosing new problems. The serious disadvantages 
of the questionnaire are: the representativeness and accuracy of replies, and 
incomplete returns. A strong antagonism toward questionnaires has been created 
because too many are distributed, often indiscriminately, Preparer of the 
questionnaire should: carefully analyze and visualize the subject, request only 
information that respondents can furnish, avoid objectionable questions, impartially 
and clearly phrase questions, ask for information that can be easily furnished by 
the respondent, arrange questions in logical sequence, and use brevity that is 
consistent with a thorough treatment of the subject. 

EVALUATION: Excellent reference worth consulting for an overall picture of 
questionnaire research. 
KEYWORDS: Applications 

History 

Advantages 

Questionnaire: design 

Textbook. 


67. Gerberich, John B. 
1947. A study of the consistency of informant responses to questions in a 
questionnaire. J. Educ. Psychol. 38: 299-306. 


Three forms of the same questionnaire were given to each of three groups 
which totaled 1,000 university students. The three forms were administered at 
different times and the results were: group A had 91.01 percent of answers con- 
sistent at 1-day intervals; group B, 76. 08-percent at 7-day intervals; and group 
C, 73.72-percent at 10-day intervals. Answers to factual questions showed less 
consistency than either the adjustment or attitude types. Sex, age, and class rank 
were not significant factors. No informant had all answers consistent, but 
veterans were slightly more consistent than nonveterans, and students who grad- 
uated in the upper third of their class were more consistent than students who 
ranked in the lower third. 

EVALUATION: Excellent paper on questionnaire validity. 
KEYWORDS: Reliability 
Validity 
Attitudes. 
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Gerberich, J. B., and J. M. Mason 
1948. Signed versus unsigned questionnaire. J. Educ. Res. 42(2): 122-126. 


A 40-item questionnaire on academic background, plans, and study habits 
was given to 2,876 students taking a biological science course. Results were 
analyzed to determine the effect of signatures upon response. No significant 
differences were found between signed and unsigned questionnaires. Questionnaire 
included. 


EVALUATION: Well-designed study; authors are careful not to overgeneralize 
conclusions. 


KEYWORDS: Anonymous respondent, 


Goldstein, Hyman, and Bernard H. Kroll 
1957. Methods of increasing mail response. J. Marketing 22(1): 55-57. 


A questionnaire survey of 4, 716 general hospitals brought total returns from 
99.7 percent of the hospitals. The original mailing of two copies of the question- 
naire plus a return envelope brought a 68.1-percent return. The first and second 
followups plus a note on colored paper: requesting cooperation had percentage 
returns of 63.6 and 56.3, respectively. The third followup using airmail and 
special delivery postage brought in a return of 72.8 percent. Although at least 
part of the high return was due to the methods employed, certain responses may 
have resulted from a sense of obligation to reply to governmental agencies' 
questionnaires. 


EVALUATION: Case study example with no control over variables that had 
greatest and least influence over response rate. 
KEYWORDS: Followup: mail 

Incentive: postage, motivation. 


Green, Bert 
1954. Attitude measurement, p. 335-369. In Gardner Lindzey (ed.), 
Handbook of social psychology. Vol. I. Cambridge, Mass.: 
Addison-Wesley Publ. Co., Inc. 


Chapter 9 deals with scaling instruments to measure attitudes. Question- 
naires, attitudes, and scales are defined along with an evaluation of reliability, 
validity, and unidimensionality of scales. Most of the chapter is a highly technical 
discussion, with mathematical equations, of scaling methods. 

EVALUATION: This chapter is most valuable for those developing or utilizing 
scales. 
KEYWORDS: Reliability 

Validity 

Sealing 

Textbook. 


Gullahorn, John T., and Jeanne E. Gullahorn 
1959. Increasing returns from non-respondents. Publ. Opin. Quart. 23: 
119-121. 


Former Fulbright and Smith-Mundt grantees in nine Midwestern States were 


72. 


73. 


sent a prequestionnaire letter asking for post-card returns with their current 
address. Only 21 percent did not respond. A sample of 20 percent of these 
grantees, including both respondents and nonrespondents was interviewed; later 
all the grantees were sent a questionnaire with a business-reply envelope. Approx- 
imately 61 percent responded. A followup questionnaire brought an additional 
18-percent response. Of the grantees who had responded to the prequestionnaire 
letter, 89 percent returned their questionnaires. Only 49 percent of those who 
had not acknowledged the first letter answered the questionnaire. Personal con- 
tact stimulated further cooperation since 95 percent of the grantees who had been 
interviewed were among the questionnaire respondents. After a second followup 
was sent, with half of the remaining nonrespondents receiving their followup 
letters by regular mail and half by special delivery, approximately half of the 
nonrespondents answered bringing the total questionnaire returns to 90 percent. 
A significantly greater proportion of the special delivery group (62 percent) than 
of the regular mail group (35 percent) responded to this followup. The use of a 
special delivery followup is worth the added expense, particularly in eliciting 
responses from individuals who have not acknowledged any previous correspondence. 
Among those grantees who had not previously responded, 60 percent of those whose 
questionnaires were mailed by special delivery answered, contrasted with 24 per- 
cent of those whose questionnaires were sent by regular mail. 
EVALUATION: Good experimental evidence. Sample size not given. 
KEYWORDS: Prequestionnaire letter 

Incentive: postage 

Followup: mail, personal contact. 


Gullahorn, Jeanne E., and John T. Gullahorn 
1963. An investigation of the effects of three factors on response to mail 
questionnaires. Publ. Opin. Quart. 27: 294-296, 


While surveying former scholarship grantees, the authors investigated the 
effects of three factors on questionnaire response frequencies: the class of 
mailing, the color of the questionnaire, and the type of postage on the return 
envelope. By means of a 2 by 2 factorial design, the sample was randomly divided 
into eight treatment groups, each receiving a different combination of the three 
factors. First-class mailing elicited significantly higher returns than third-class 
(51 vs. 49 percent), but the color of the questionnaire was insignificant. Stamped 
return envelopes were significantly more effective than business-reply enclosures 
(52 vs. 48 percent). 

EVALUATION: Very small but statistically significant results. 
KEYWORDS: Incentive: postage, motivation. 


Hamel, LaVerne, and Hans G. Reif 
1952. Should attitude questionnaires be signed? Personnel Psychol. 5(2): 
87-91, illus. 


A total of 109 identified and 94 anonymous employee attitude questionnaires 
were obtained from department store employees. The questionnaire probed 
working conditions, supervision, coworkers, communication, company hours, 
pay, and type of work. Mean scores for the two groups did not differ significantly. 
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Results may have been influenced by the fact that questionnaires were administered 
by a university staff and respondents were assured that the questionnaires were 
for confidential research purposes only. 

EVALUATION: In the absence of any mention about nonresponse, one wonders if 
nonresponse bias could affect results of this study. 

KEYWORDS: Anonymous respondent. 


Haneock, John W. 
1940. An experimental study of four methods of measuring unit costs of 
obtaining attitude toward the retail store. J. Appl. Psychol. 24(Apr.): 
213-230. 


Four methods were used to determine attitudes toward independent stores 
versus chain stores. The cost per unit of usable returns for each method were: 
mail questionnaire with accompanying letter, $1.22; mail questionnaire with letter. 
plus 25 cents, $0.79; mail questionnaire with letter and a promise that 25 cents 
would be mailed upon receipt of the questionnaire, $1.06; personal interview, 
$0.74. The questionnaire with 25 cents enclosed was the most satisfactory mail 
method with 47. 2-percent usable returns. The questionnaire and letter only pro- 
duced a response of 9.56 percent. 


EVALUATION: Brief report of a Ph. D. dissertation. Conclusions support data. 
KEYWORDS: Incentive: money 

Mail questionnaire vs. interview 

Cover letter. 


Hansen, Morris H., and William N. Hurwitz 
1946. The problem of non-response in sample surveys. Am. Statist. Assoc. 
J. 41: 517-529. 


Mail questionnaires are low cost but may have high nonresponse, whereas 
interviews give complete response but have relatively high costs. By combining 
the advantages of each method, one can interview a sample of the nonrespondents 
of a questionnaire study. Given an estimate of the relative costs and the rate of 
response, one can compute the number of questionnaires necessary for a desig- 
nated degree of reliability. With the actual rate of response, the number of 
interviews can then be adjusted to obtain this reliability. Appendix included with 
derivations for the statistical formulas. 

EVALUATION: Extensive mail followup should precede interview. Bias might 
exist between mail questionnaire and interview such as evaluation apprehension 
in face-to-face contact. 
KEYWORDS: Bias: nonresponse 

Mail questionnaire vs. interview. 


Hobson, Asher 
1916. The use of the correspondence method in original research. Am. 
Statist. Assoc. J. 15(114): 210-218, illus. 
A one-question letter is recommended for simplicity. If more questions 


are necessary, a one-page questionnaire is desirable over a more bulky presenta- 
tion. Questionnaire results should be verified from other sources if possible. 
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Questions should interest recipients. Accompanying letters should be personal, 
with names of the addressees rather than a "Dear Sir" and signatures of the 
sender rather than a rubber stamp. 


EVALUATION: A presentation of rules of thumb for mail questionnaire use which 
have changed little in light of recent quantitative evaluation of the same "rules." 


KEYWORDS: Questionnaire: design 
Incentive: motivation. 


Hochstim, Joseph R., and Demetrios A. Athanasopoulos 
1970. Personal follow-up in a mail survey: its contribution and its cost. 
Publ. Opin. Quart. 34: 69-81. 


An area probability sample was drawn of Alameda County, California, which 
consisted of 8,083 persons for whom at least demographic and socioeconomic data 
were gathered by enumeration. A questionnaire was left at each household with 
a cover letter and return envelope. Returns were secured from 6,928 persons or 
86 percent of the listed adults. Forty-two percent returned the questionnaire 
after the enumeration; 17 percent, after receiving a followup letter; 11 percent, 
after a telegram; 16 percent, when an interviewer called on them; and 14 percent 
did not respond. The nonrespondents were older and more likely to be retired, 
widowed; to live in a small household without children; to own their own home; 
to be white, male, and skilled. Minority groups were slightly overrepresented 
in the questionnaire returns due to their overrepresentation in the personal contact 
stage. Personal-contact followup tended to improve estimates associated with 
low socioeconomic status, and also resulted in some not-at-home bias resulting 
in a slight overrepresentation of women and large families. Extrapolation of the 
data could not be used to improve the estimates. People who reported no organi- 
zational membership or political activity, and people with low education, job 
insecurity, and no provision for medical care were less likely than others to 
respond by mail. The 5,630 mail responses cost about $6.55 each, and the 1,298 
questionnaires secured by interviewers cost $13.65 each. 


EVALUATION: Careful documentation of effect of various procedures in a large 
questionnaire study. 
KEYWORDS: Followup: mail, personal contact 

Cover letter 

Bias: nonresponse. 


Hoppe, Donald J. 
1952. Certain factors found to improve mail survey returns. Proc. Iowa 
Acad. Sci. 59: 374-376. 


Post cards sent to automobile owners of cars observed on the road at a 
given hour asked age, sex of driver, number of persons in car, and miles driven 
during the hour observed. Adding the sentence, ''Please do not let your failure to 


return this card make it impossible for us to obtain a 100 percent return, "' increased 


response significantly to 64.3 percent. In a second post-card survey in Minnesota 
and Iowa, adding an identification number in invisible ink versus writing a case 
number disguised in the address made no significant difference in return rates. 
EVALUATION: Study unclear as to the construct tested. 
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KEYWORDS: Questionnaire design 
Bias: nonresponse 
Incentive: motivation. 


Hubbard, Alfred W. 
1950. Phrasing questions. J. Marketing 15: 48-56. 


A respondent's experience and purpose are integrated with his perceptions 
which necessarily change a factual answer to a question into a less objective 
"opinion. "' An unbiased opinion is never obtained; therefore, the researcher 
seeks the number of persons who do or do not have such biases and the degree of 
such biases. The bias should not be influenced or changed by the question itself. 
Awareness of this problem enables the question phraser to balance the implied 
favorable and unfavorable judgments against one another. For each question, 
three preliminary questions should be written and compared side by side. Then 
pretest several variations to determine bias and re-pretest any revisions. The 
reliability of alternative questions is discussed. For example, introducing a 
negative into the question increases suggestibility and decreases reliability. 
Many other rules and examples are given. 

EVALUATION: This article should be read because it deals with seemingly minor 
but very technical differences in wording questions that are too often taken for 
granted. 
KEYWORDS: Question: wording 

Open-end questions 

Reliability. 


Hubbard, Frank W. 
1939. Questionnaires. Rev. Educ. Res. 9(5): 502-507. 


A review of five articles on questionnaire reliability, seven articles on 
validity, two on rate of return, and one on the use of illustrated questionnaires. 
Three trends and innovations include illustrated questionnaires, opinion-type 
questionnaires, and questionnaires asking the recipient if he will participate in 
a more detailed study. 

EVALUATION: Dated but concise and well-organized review of questionnaire 
literature. 
KEYWORDS: Reliability 

Validity 

Bias: nonresponse 

Literature review. 


1942. Questionnaires, interviews, personality schedules. Rev. Educ. Res. 
12(5): 534-541. 

A general discussion of the literature appearing before 1942 which pertains 
to questionnaire wording, pretesting, sampling, followup, reliability, and use. 
EVALUATION: Brief with little substantive content, but valuable as a bibliographic 
reference. Bibliography, 56. 

KEYWORDS: Literature review 
Mail questionnaire vs. interview. 
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Jahoda, Marie, Morton Deutsch, and Stuart W. Cook 
1951. Data collection: the questionnaire and interview approach, p. 151-208. 
In Research methods in social relations with special reference to 
prejudice. New York: Dryden Press. 

. Chapter 6 includes questionnaires and interviews as sources of data, a 
comparison of the interview and questionnaire technique, types of question content, 
types of interviews and questionnaires, the sociometric method and pictorial tech- 
niques of data collection, questionnaires which form attitude scales, and the use 
of rating scales. 

EVALUATION: Excellent reference. 
KEYWORDS: Advantages of questionnaires 
Bias: representativeness 
Disadvantages of questionnaires 
Open-end questions 
Question: content, wording 
Scaling 
Textbook. 


Jourard, Sidney, and Paul Lasakow 
1958. Some factors in self-disclosure. J. Abnorm. & Soc. Psychol. 56(1): 
91-98, illus. 


A reliable questionnaire for the assessment of self-disclosure was used to 
test groups of both sexes, white and Negro, married and unmarried, for extent of 
self-disclosure of six different aspects of self to various target-persons: mother, 
father, male friend, female friend, female friend and spouse or both. The findings 
are summarized as follows: Young unmarried subjects, both white and Negro, 
showed the highest self-disclosure to mother. Subjects tended to vary the amount 
of self-disclosure with respect to the category of information to which an item 
about the self belonged. Two clusters of aspects emerged: a high disclosure cluster 
including attitudes and opinions, tastes and interests, and work; and a low disclosure 
cluster comprised of money, personality, and body. White subjects disclosed 
more than Negroes, and females more than males. There was significant inter- 
action among groups of subjects, target-persons, and aspects of self. Married 
subjects disclosed less to mother, father, and same-sex friend than comparable 
unmarried subjects. Those who were married disclosed more to their spouses 
than to any other target-person. A significant correlation was found between 
parent-cathexis and self-disclosure to the parents. The more that the parents 
were liked, the more disclosures were made to them. 

EVALUATION: Of only indirect interest to the use of questionnaires since focused 
on question of psychological concern. Identifies differences in self-disclosure 
that may have implications for questionnaire research, 
KEYWORDS: Validity 

Reliability 

Bias: stated vs. actual behavior. 


Katz, Daniel 
1946. The interpretation of survey findings. J. Soc. Issues 2: 33-44, 
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Surveys should include: (1) explicitly stated assumptions, (2) research 
instrument designed on well-constructed hypotheses, (3) selection of an appropriate 
sample, (4) methods appropriate to study problems, and (5) determination of 
opinions you are sampling. Errors to be avoided are: (1) failure to ask related 
and dependent questions, (2) failure to determine respondent's frame of reference, 
(3) substitution of absolute value for percentage statistics, (4) quotation of a 
single respondent as "typical", (5) misuse of the "‘quota-control"’ method of 
sampling to control for the representativeness of subgroups. 

EVALUATION: Good general discussion of possible pitfalls in interpreting survey 
research findings. 
KEYWORDS: Validity 

Questionnaire: design 

Interpretation. 


Kawash, Mary B., and Lawrence M. Aleamont 
1971. Effect of personal signature on the initial rate of return of a mailed 
questionnaire. J. Appl. Psychol. 55(6): 589-592, illus. 


A mail questionnaire was used to test if initial return rates would be higher 
for those receiving a personal signature on the cover letter than for a mimeo- 
graphed facsimile. Three thousand students and ninety-one faculty members 
of the University of Illinois were asked about their use of and attitudes toward audio- 
visual instructional materials. No difference was found between those receiving the 
personal signature and those receiving the mimeographed one even when faculty were 
subdivided on rank (professor, associate professor, assistant professor, etc.). 
EVALUATION: It is difficult to generalize findings to other studies because of 
uncontrolled variables operating in the nonresponse group. For example, no 
attempt was made to control for those who did not use the audiovisual facilities 
available, and therefore felt the questionnaire did not apply to them. 


KEYWORDS: Bias: nonresponse 
Cover letter. 


Kelly, Robert Lincoln 
1931. The revolt against questionable questionnaires. Assoc. Am. Coll. 
Bull. 17(3): 377-390. 


Article proposes a plan by the Association of American Colleges for regu- 
lating the use of "poorly constructed and meaningless" questionnaires sent to 
college administrators. Outlined are questions asked about proposed question- 
naires to determine if they should be answered. These include: Is the question- 
naire adequately sponsored? Is its purpose stated? Is it a worthy educational 
topic? Can it be answered briefly? What guarantees are given that the information 
will be put to good use? 


EVALUATION: Unlike the majority of antiquestionnaire articles of this era, the 
article offers questions that would be valuable to anyone constructing his own 
questionnaire. 
KEYWORDS: Questionnaire: design 

Disadvantages of questionnaires. 
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Kephart, William M., and Marvin Bressler 
1958. Increasing the responses to mail questionnaires: a research study. 
Publ. Opin. Quart. 22: 123-132. 


A questionnaire was sent to 100 nurses divided into 10 groups, but each 
group received different incentives to affect the return rate. These included a 
questionnaire only; a preview letter and then the questionnaire; a questionnaire 
plus a followup letter; a preview letter, questionnaire, and followup letter; 
questionnaire sent airmail; questionnaire sent special delivery; and questionnaire 
plus either a penny, nickel, dime, or quarter. Fifty-two percent returned the 
regular questionnaire, but the quarter incentive inspired a 70-percent return. 
The airmail delivery produced a 60-percent return; followup, 68-percent; and 
the special delivery, 66-percent. Other inducements had little effect. The 
inducements themselves may create a biased return. In view of the lower cost 
of followup procedures over money incentives, the former is the recommended 
procedure. 

EVALUATION: Clear presentation of a large variety of inducements to increase 
mail returns. Worth reading before considering special incentives. 
KEYWORDS: Prequestionnaire letter 

Followup: mail 

Incentive: money, postage. 


Kivlin, Joseph E. 
1965. Contributions to the study of mail-back bias. Rural Sociol. 30(3): 
323-326. 


Study based on 487 Pennsylvania farmers to determine effect on certain 
attributes of the rate of adoption of improved farm practices. Data show signifi- 
cant differences between respondents and nonrespondents. The nonrespondents 
adopted fewer practices, were older, had less education, and were less likely 
to belong to organizations. Perhaps more important than the differences is 
evidence that these differences may not seriously affect relationships among the 
variables being investigated. Mail-back bias appears broad and cuts across 
socioeconomic and research variables. The data also support the hypothesis that 
differences between early and late respondents tend to be indicative of differences 
between respondents and nonrespondents, but these can only be considered 
inferences. 

EVALUATION: Paper worth consulting on the nonresponse issue. 
KEYWORDS: Bias: nonresponse 
Early vs. late response. 


Knox, John B. 
1951. Maximizing responses to mail questionnaires: a new technique. 
Publ. Opin. Quart. 15: 366-367. 


Questionnaires were mailed to 173 unemployed Tennessee railroad firemen 
who were localized and suspicious of written communication which might adversely 
affect their chances of railroad reemployment. After 29.5 percent were returned, 
a followup letter and another copy of the questionnaire were sent. To further 
encourage replies, a turkey was offered to the lucky respondent whose number was 
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drawn. A postal authority stated that the practice used here was not a lottery and 
not illegal since the object was not financial profit. The response to this followup 
letter and prize offer was 18.8 percent. 
EVALUATION: Very brief procedural account of a survey ignoring final results 
on the effectiveness of the technique, similar studies with conflicting results, 
nonresponse bias, etc. Of little value. 
KEYWORDS: Incentive: reward 

Followup: mail. 


90. Knudsen, Dean D., Hallowell Pope, and Donald P. Irish 
1967. Response differences to questions on sexual standards: an inter- 
view- questionnaire comparison. Publ. Opin. Quart. 31: 290-297. 


Three different samples of white women, all of whom were or had been 
premaritally pregnant for the first time, were used. The first sample, 150 pri- 
vate patients of physicians, anonymously completed questionnaires in their physi- 
cians' offices. The second sample, 311 women selected from birth certificates, 
was interviewed confidentially. The third sample, 77 women in maternity homes, 
filled out questionnaires in the presence of an interviewer. Four similarly worded 
questions were asked of each sample. Data suggest that in interview situations 
respondent was more likely to support the public and restrictive sexual norms 
that she assumed were adhered to by the interviewer. Lower socioeconomic 
respondents deferred to the norms represented by the higher status interviewers. 
In the private and anonymous questionnaire situation, the respondents more often 
answered to subcultural norms. Therefore, compared with anonymous question- 
naires, interviews may lessen the expression of deviance. 

EVALUATION: Good experimental documentation of anonymity benefits of 
questionnaires. 
KEYWORDS: Mail questionnaire vs. interview 

Anonymous respondent. 


91. LaPiere, Richard T. 

1934. Attitudes vs. actions. Soc. Forces 13(2): 230-237. 

Social attitudes are conditioned responses to social stimuli but, by derivation, 
are seldom more than verbal responses to symbolic situations. All measurement 
of attitudes by questionnaires proceeds on the assumption that there is a mechani- 
cal relationship between symbolic and nonsymbolic behavior. An unusual study 
indicated that this assumption is unreliable. While traveling with a foreign-born 
Chinese couple, the author and his companions purchased, in 251 instances, goods 
or services necessitating intimate human relationships. Only in one instance was 
service refused. It was concluded that quality and condition of clothing, etc., 
influenced the behavior of others toward the Chinese more than did race. After 
6 months elapsed, the establishments patronized were surveyed by questionnaire. 
Thus, the overt reaction and the symbolic were both obtained. The questionnaires 
asked, ''Will you accept members of the Chinese race as guests in your establish- 
ment?'' Out of 128 replies, 92 percent of the restaurants and 91 percent of the 
hotels said ''No."' The remainder replied, "Uncertain, depends on circumstances." 
Only one "yes'' was received. The author states that "social attitudes must be 
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derived from a study of humans behaving in actual social situations, not on the 
basis of questionnaire data.'' The questionnaire is cheap, easy, mechanical, 
and gives quantitative results. Human behavior study is time consuming, intellec- 
tually fatiguing, and dependent on the ability of the investigator. Its results are 
qualitative. Quantitative measurements are quantitatively accurate, qualitative 
evaluations are subject to human errors. Yet it would seem more worthwhile 
to make a shrewd guess regarding that which is essential than to accurately 
measure that which is likely to prove irrelevant. 
EVALUATION: An interesting, empirical comparison of observations versus 
questionnaires that challenges the validity of questionnaire responses to predict 
behavior. 
KEYWORDS: Advantages of questionnaires 

Disadvantages of questionnaires 

Validity 

Bias: stated vs. actual behavior. 


Larson, Richard F., and William R. Catton, Jr. 
1959. Can the mail-back bias contribute to a study's validity? Am. Sociol. 
Rev. 24(2): 243-245. 

Bias can have a "favorable" effect when selective returns are desired. An 
eight-page questionnaire, a stamped return envelope, cover letter, post card, 
and two followup letters were used in a study of 700 persons named on the mailing 
list of a national organization. A comparison of characteristics of early respon- 
dents to respondents of successive waves of followup and nonrespondents suggests 
that the latter group includes persons for whom the organization was not a signifi- 
cant reference group. 

EVALUATION: Good systematic evidence on the nature of nonresponse bias. 


KEYWORDS: Bias: nonresponse 
Incentive: motivation 
Followup: mail, telephone. 


Lawson, Faith 
1949, Varying group responses to postal questionnaires. Publ. Opin. 
Quart. 13: 114-116. 


A sample of 950 persons in Great Britain was stratified by occupation and 
sent a questionnaire measuring attitudes toward gambling. In the 23-percent 
returns obtained, responding most frequently were psychologists, followed by 
secretaries, clergymen, lawyers, solicitors, trade union officials, musicians, 
members of Parliament, and bookmakers. Interest in gambling was considered 
the determining factor affecting response. Questionnaires from lawyers, book- 
makers, and trade union officials were received mostly within the first week. 
EVALUATION: The extremely low return rate makes conclusions questionable. 
Response did support contention that interest in questionnaire topic affects 
response. 

KEYWORDS: Bias: nonresponse 
Incentive: motivation. 
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Lazarsfeld, Paul F. 
1940. The use of mail questionnaires to ascertain the relative popularity 
of network stations in family listening surveys. J. Appl. Psychol. 
24(Spring): 802-816. 


Direct interview and mail questionnaire methods for obtaining information 
are compared. Respondents to 669 mail questionnaires were later interviewed 
for the same information. Only minor differences were discovered in their res- 
ponse. Nonrespondents to the questionnaire had lower literacy and interest in 
the topic than respondents. The questionnaire is likely to be answered by the 
husband; the interview, by the wife. 

EVALUATION: Logical article with empirical data on the use of questionnaires 
versus interviews, and nonresponse bias as a function of respondent interest 
and literacy. 
KEYWORDS: Mail questionnaire vs. interview 

Bias: nonresponse. 


Lehman, Edward C., Jr. 
1963. Tests of significance and partial returns to mail questionnaires. 
Rural Sociol. 28: 284-289, illus. 


Observable differences between early and late respondents appears to pro- 
vide the basis for approaching problems of nonresponse with regard to representa- 
tiveness, sample probabilities, and the use of tests of significance. The technique 
suggests the nature of nonrespondents by indicating what effect they would have on 
the analysis hac they responded. The method can provide qualitative information 
for general interpretation and can suggest differences between returns and non- 
responses and their direction. 

EVALUATION: Excellent article that should be consulted before analyzing data. 
The best the approach gives is a prediction of nonrespondent characteristics. 
KEYWORDS: Bias: nonresponse 

Early vs. late response. 


Levine, Sol, and Gerald Gordon 
1959. Maximizing returns on mail questionnaires. Publ. Opin. Quart. 22: 
568-575. 

Within 2-1/2 months of the mailing, 100-percent returns were obtained of 
two different questionnaires to each of 85 Blue Cross plans. A 90-minute question- 
naire was sent to the enrollment director and a 20-minute questionnaire to the 
executive director of each plan. This article consists of suggestions to other 
researchers based on experience in the study; e.g., respondent preparation and 
involvement, questionnaire design and construction, and various followup proce- 
dures. A wealth of how-to-do-it questionnaire information is included, although 
the respondents in the study may have felt obliged to respond due to their official 
responsibilities rather than the author's expert technique. 


EVALUATION: Good how-to-do-it information. 
KEYWORDS: Questionnaire: design 


Followup: telephone, mail 
Incentive: motivation. 
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Lewin, Kurt 
1951. Problems of research in social psychology, p. 163. in Field theory 
in social science. New York: Harper. 


Discusses in one paragraph the use to which questionnaire data should be put 
by the researcher. Author suggests treating questionnaire answers not as an 
expression of fact but as reactions to a situation which are partly determined by 
the question and partly by the general situation of the respondent. 

EVALUATION: Very brief statement of a perspective worth considering. 
KEYWORDS: Interpretation 
Textbook. 


Lindsay, E. E. 
1921. Questionnaires and follow-up letters. Pedagog. Seminary 28(3): 
303-307, illus. 
Two methods of timing followup letters were tested on 2,782 respondents: 
(1) plotting the returning questionnaires and sending out followups when the effects 
of the previous letter dropped and (2) sending followups soon after the original 
questionnaire and from then on at slightly increasing intervals. The first method 
resulted in 68-percent return and the second method in 92-percent return. Author 
concludes that questionnaire followup increases returns, with the second method 
the better of the two. 
EVALUATION: Methodologically weak in that return rates are not comparable 
since the two followup techniques involved a different questionnaire and sampling 
universe. Nonetheless, the two methods are useful for timing followup letters. 
KEYWORDS: Bias: nonresponse 
Followup: mail. 


Linn, Lawrence S. 
1965. Verbal attitudes and overt behavior: a study of racial discrimination. 
Soc. Forces 43(3): 353-364, illus. 

To measure the relationship between racial attitudes and overt behavior, 
subjects were asked to pose for a photograph with a Negro of the opposite sex. 
Discrepancies between verbal attitudes and subsequent behavior were found in 
59 percent of the cases. The relationship between attitude (prejudice) and behavior 
(discrimination) is seen to be a function of the level of social involvement with the 
attitude object as well as the amount of prior experience with it. One implication 
of the study is that statements or predictions of racial behavior based on attitude 
measurements have little reliability unless first validated empirically. 
EVALUATION: Long article also reviews several other studies showing discrep- 
ancy between questionnaire-measured attitudes and overt behavior. 

KEYWORDS: Literature review 
Disadvantages of questionnaires 
Validity 
Bias: stated vs. actual behavior. 


Linsky, Arnold S. 
1965. A factorial experiment in inducing responses to a mail questionnaire. 
Sociol. & Soc. Res. 49(2): 183-189. 
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Experimental study tested the contents of cover letters as they relate to 
response rate. Questionnaires were sent to 912 nurses; only the cover letters 
differed in the 16 different combinations of four variables: (1) handwritten per- 
sonal salutation vs. 'Dear member" and a mimeographed signature; (2) presence 
and absence of an explanation of the place and importance of respondent; (3) pres- 
ence and absence of an argument for the social utility of the research; (4) presence 
and absence of an appeal to help the study researchers. Response rates signifi- 
cantly increased when a personalized letter and the place and importance of the 
respondent were included. Results showed that personalization and importance 
of the respondent brought higher returns (56.1 percent) than when all four factors 
were present (43.9 percent). 

EVALUATION: Excellent study giving empirical evidence and 22 references. 
KEYWORDS: Cover letter 
Incentive: motivation. 


Longworth, Donald S. 

1953. Use of a mail questionnaire. Am. Sociol. Rev. 18(3): 310-313. 

In each of a series of pretests, 50 randomly selected respondents answered 
an eight-page questionnaire. Placing small denomination stamps of various colors 
on the envelope increased returns by 2 percent. A personal note and typed letter 
of explanation on letterhead paper increased returns by 5 percent. When a news- 
paper clipping about the study was included, a further increase of 5 percent was 
noted. The followup phone call increased returns by 37 percent. In the study 
that followed the pretests, 800 questionnaires were distributed, and a total of 
69 percent were returned. 

EVALUATION: Good experimental evidence on effect of certain procedures on 
questionnaire returns. 
KEYWORDS: Incentive: postage, motivation 

Cover letter 

Followup: telephone. 


Lundberg, George A. 
1929. Social research. Chapters 6 and 9. New York: Longmans, Green 
and Co. 


These chapters discuss the value of systematic measurement through 
questionnaires, thoroughly describe their construction (size, shape, material, 
color, arrangement of items, wording), and make suggestions for getting accurate 
and wide response. Eight general types of questionnaires which can be used to 
measure attitudes are described and illustrated. Validity and the limitations of 
measurement in general are discussed. 

EVALUATION: Although old, this book appears highly useful. Chapter 6 in 
particular might be consulted when designing a questionnaire study. 
KEYWORDS: Questionnaire: design 

Applications 

Mail questionnaire vs. interview 

Advantages of questionnaires 

Validity 

Disadvantages of questionnaires. . 
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Macoby, E. A., and Nathan Macoby 
1954. The interview: a tool of social science. Handbook of social psychol- 
ogy I: Theory and method. Reading, Mass.: Addison-Wesley. 


This handbook deals explicitly with the methodology of interviewing but is 
directly applicable to mail questionnaires. Included are discussions on: when 
open-end questions are preferable to closed-end; how to word questions to pro- 
mote understanding, accuracy, and conceptual validity; modifications and pre- 
cautions necessary for cross cultural or linguistic settings; presentation of 
research findings on the validity of questionnaire versus interview data; four 
ways of judging the validity of the findings. Advantages and disadvantages of 
interviews and mail questionnaires, and when each is used best are included. 
EVALUATION: Comprehensive, especially relevant where the choice of method 
is a concern. 

KEYWORDS: Validity 
Applications 
Questionnaire: design 
Mail questionnaire vs. interview 
Textbook. 


Magid, Frank N., Nicholas G. Fotion, and David Gold. 
1962. A mail-questionnaire adjunct to the interview. Publ. Opin. Quart. 
26: 111-114. 


The mail questionnaire can be used as an adjunct to the interview to shorten 
interview time without diminishing the volume of information. Nonresponse bias 
can be estimated from the distribution of related variables identified in the inter- 
view which characterize respondents and nonrespondents. Example of this is 
included. The rapport, interest, and commitment built up during the ‘interview 
increase the probability of questionnaire response. Different interviewers 
successfully solicit different response rates to followup questioning. Other 
advantages include the provision of more anonymity and the possibility of making 
reliability checks to determine if differences between the individual interviewers 
can be explained by differences in the characteristics of their respondents. 
EVALUATION: A descriptive how-to-do-it article by survey consultants. 
KEYWORDS: Applications 

Bias: nonresponse 
Mail questionnaire vs. interview. 


Manfield, Manuel N. 


1948. A pattern of response to mail surveys. Publ. Opin. Quart. 12: 
493-495, 


The longer a recipient delays in answering, the less likelihood he will 
ever respond. Respondent interest in the topic of the questionnaire and his 
education level affect the rate of return but are of secondary importance. 
Veterans Administration questionnaires show roughly one-half of all returns 
to a particular mailing are received by the 3d day, three-fourths by the 5th day, 
and nine-tenths by the 10th day. Tabulation of returns should start the day the 
first batch of returns is received, and reports should include size of mailing, 
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number of returns, and description of sample. 
EVALUATION: Superficial but illustrates rate and speed of questionnaire returns. 
KEYWORDS: Bias: nonresponse. 


Mason, Ward S., Robert J. Dressel, and Robert K. Bain 
1961. An experimental study of factors affecting response to a mail survey 
of beginning teachers, Publ. Opin. Quart. 25: 296-299. 


Differences in response rate were tested for combinations of a six-page, 
62-item questionnaire; an eight-page, 92-item questionnaire; an address with 
respondent's name and code number; and an address with code number only. 
Four forms were randomly distributed to 741 teachers, and no significant differ- 
ences in response were found. Neither placing the respondent's name and 
address directly on the questionnaire nor the use of a longer questionnaire 
influenced the results. 


EVALUATION: Brief and to the point. 

KEYWORDS: Questionnaire: length 
Anonymous respondent 
Bias: nonresponse. 


Mayer, Charles S., and Robert W. Pratt, Jr. 
1966. A note on nonresponse in a mail survey. Publ. Opin. Quart. 30: 
637-646. 


Data were collected as part of a study designed to explore the economic 
and psychological consequences of personal injury automobile accidents in 
Michigan. Returns were received from 2, 137 of the 2, 872 individuals (74 percent) 
receiving questionnaires. When nonrespondents were segmented into two 
categories, those who refused an interview and those who could not be located, 
each category produced a different, and sometimes offsetting, bias. Respondents 
who cannot be located tend to differ from respondents in that they are more often 
male, younger, and nonwhite, and they possess lower occupational skills. When 
these factors are combined, they represent a relatively mobile group in society. 
Analysis shows serious bias when that portion of the sample which has moved 
residence and does not receive a questionnaire is subtracted from the total sample 
size in order to show a higher percent return. If individual involvement in the 
survey subject underlies the motivation to respond, motivation provides a means 
of predicting the distribution of the characteristics of those who refuse. Biases 
introduced by nonrecipients of a questionnaire tend to coincide with characteristics 
of the mobile portion of the population being studied. As long as the relative size 
of nonresponse groups is known and the directions of bias can be evaluated 
through knowledge about the motivations of the ''refusers" and the characteristics 
of the "mobiles, '' meaningful techniques can be developed to adjust for possible 
nonresponse bias. 


EVALUATION: Excellent paper worth consulting before running an analysis. 
KEYWORDS: Bias: nonresponse 


Early vs. late response 
Incentive: motivation. 
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Mayer, Edward N., Jr. 
1946. Postage stamps do affect results of your mailing. Printers' Ink 
PUK (OLE, VB Bal 
If all other factors are equal, the best return rate results from colored 
stamps in the following order: purple, blue, red, brown, green, and black. 
Newly issued stamps increase returns regardless of color. First class over 
third class mailing may or may not result in more returns. 


EVALUATION: Weak. Lacks supporting data for the results presented. 
Business advertising is the probable source of facts. 


KEYWORDS: Incentive: postage. 


McDonagh, Edward C., and Leon A, Rosenblum 
1965. A comparison of mailed questionnaires and subsequent structured 
interviews. Publ. Opin. Quart. 29: 131-136. 


Questionnaires were mailed to 2,497 randomly selected residents listed 
in a California city directory. Interviews were conducted on a 10-percent 
subsample of the 1,117 respondents and the 1,262 nonrespondents. The data 
show no statistically significant differences between the mailed questionnaire 
and the structured interview. The mailed questionnaire revealed representative 
responses in spite of the partial return. 

EVALUATION: Important methodological study suggesting choice of data 
gathering technique can be based on the type of information desired rather than 
representativeness of results. 
KEYWORDS: Bias: nonresponse 

Mail questionnaire vs. interview. 


McNemar, Quinn 
1946. Opinion--attitude methodology. Psychol. Bull. 438(4): 289-374. 


A critical appraisal of major methodological problems involved in study 
of opinions and attitudes. Topics include problems and issues, attitudes by 
scaling techniques, single-question opinion gaging, administration, statistical 
issues, study of changes, correlates and interrelationships, and studies of 
morale. Stresses need for greater reliability, validity, and unidimensionality 
of opinion- and attitude-measuring instruments. Questions the dependability of 
single-question measures and recommends the use of attitude scales to attain 
satisfactory reliability. Unitary scales can be developed using Guttman scaling 
technique. Validity can be established, but not without more effort than is 
usually expended. Study of attitude change might be conducted under experi- 
mental conditions with control groups and study of attitude and opinion correlates, 
but greater stress on formulation of hypothesis is needed. 

EVALUATION: Dated, but a definitive "state of the art" work in opinion and 
attitude measurement with excellent literature review (133 citations). 
KEYWORDS: Reliability 

Validity 

Literature review 

Attitudes 

Sealing, 
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Metzner, Charles A. 
1950. An application of scaling to questionnaire construction. Am. 
Statist. Assoc. J. 45(249): 112-118, illus. 


The best form of a question on the price World War II veterans would pay 
for housing was determined by Thurston scaling. Semantically different ques- 
tions were submitted to 100 people who judged the meaning of the questions. 
The question most central to the scale presumably had the least bias. The 
undesirable semantic effects of words or phrases were negated in the question- 
naire to the veterans. 

EVALUATION: A good example of the possibilities which scaling techniques 
can offer for question construction. 
KEYWORDS: Questionnaire: design 

Sealing 

Question: wording. 


Metzner, Helen, and Floyd Mann 
1952. <A limited comparison of two methods of data collection: The fixed 
alternative questionnaire and the open-ended interview. Am. Sociol. 
Rev. 17(4): 486-491. 


About 800 company employees filled out fixed alternative questionnaires 
and were interviewed to determine if differences existed between the two data 
collecting methods on personal and background characteristics and feelings about 
their jobs. Respondents indicated more positive attitudes toward their jobs on 
interviews than on questionnaires. The degree of confidence in anonymity and 
degree of satisfaction with job might account for the differences. 

EVALUATION: Well-written article relevant to selecting a research instrument 
for measurement of attitudes. 
KEYWORDS: Mail questionnaire vs. interview 

Bias: stated vs. actual behavior 

Anonymous respondent. 


Mitchell, Walter, Jr. 
1939. Factors affecting the rate of return on mailed questionnaires. Am. 
Statist. Assoc. J. 34(208): 683-692, illus. 


Seven factors affect the rate of returns: selectivity of the mailing list, 
timeliness, prestige of the sending agency, salesmanship in the cover letter, 
reluctance of respondents to identify confidential figures, respondents' stake in 
the success of the survey, and time in filling out the questionnaire. Forty-three 
surveys are plotted to show how time burden affects the percent return. Best 
returns are obtained when questionnaires are brief, on a current interest topic, 
sponsored by a prestigious organization, and sent to large enterprises who 
have previously cooperated. 


EVALUATION: Good attempt to quantify factors related to rate of return. 
KEYWORDS: Incentive: motivation 


Bias: nonresponse 
Questionnaire: format. 


114, 


115. 


116. 


Moore, Clarence Carl 
1941. Increasing the returns from questionnaires, J. Educ. Res. 35(2): 
138-141. 


Questionnaires were sent to 494 school superintendents, Of the accompany- 
ing letters, 238 were individually typewritten and 256 were duplicated. A 
duplicated followup letter was sent 8 weeks later. Eighty-two percent of the 
typewritten sample responded whereas 66 percent of the duplicated letter group 
responded. Conclusion is that typewritten letter accounted for 16-percent increase 
in number of questionnaires received. Followup is also effective in increasing 
response. 


EVALUATION: The followup should have been typewritten or duplicated to be 
consistent with the original mailing. Quality of the duplication may greatly 
affect results also. 


KEYWORDS: Followup: mail 
Incentive: motivation. 


Morgan, Roy 
1948, Follow-up letters disclose trends following opinion surveys. Publ. 
Opin. Quart. 13: 686-688. 


Australian public opinion polls commonly send followup letters to people 
interviewed in regular opinion surveys to check the performance of interviewers 
and to reveal shifts in opinion which take place after a survey. The letters 
reproduce the answers which respondents gave a month previously and ask 
respondents to indicate the cases in which their opinions have changed. Business 
reply envelopes are enclosed, and returns average between 50 and 55 percent 
of all letters sent out. In general, respondent opinions on most subjects are 
stable, but there is a uniform decrease in the "undecided" or "no opinion" 
categories for mail surveys. This latter effect suggests that mail surveys make 
respondents think about current problems. 

EVALUATION: Poor mail return leaves half the sample out of the analysis. 
KEYWORDS: Applications 

Mail questionnaire vs. interview 

Followup: mail. 


National Education Association 
1930. The questionnaire. Res. Bull. Natl. Educ. Assoc. 8(1): 1-51, illus. 


Contains reasons for and against questionnaire use, ways to prepare 
better questionnaires, ways to report questionnaire results, a proposed plan 
for the cooperative regulation of questionnaires, and 33 references. 
EVALUATION: Useful "handbook" for questionnaire research even though it is 
compiled for school administrators. 


KEYWORDS: History 
Disadvantages of questionnaires 


Advantages of questionnaires 
Literature review 
Questionnaire: design, format, 
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Nichols, Robert C., and Mary Alice Meyer 
1966. Timing postcard follow-ups in mail-questionnaire surveys. Publ. 
Opin. Quart. 30: 306-307. 


A 12-page questionnaire was sent to 1,600 college students. Half received 
a followup post card 3 days after the questionnaire was mailed; the other half 
did not. An additional followup post card was sent to half of each group after 
16 days and to the other half 27 days after the questionnaire was mailed. After 
45 days, a second copy of the questionnaire was sent to all remaining nonrespon- 
dents. At 120 days, the response rate was significantly higher for the 3-day 
post-card group. After 45 days, there were no Significant differences in res- 
ponse rate between the 16-day followup and the 27-day followup. The best 
response rate was a combination of 3-day and 27-day followups which by the 27th 
day produced a 77- to 51-percent advantage over the group that received no 
followup. 
EVALUATION: Excellent reference pointing to the need for followup reminders. 


KEYWORDS: Followup: mail 
Incentive: postage. 


Nixon, John E. 
1954. The mechanics of questionnaire construction, J. Educ. Res. 47(7): 
481-487. 


Presents a series of practical suggestions for constructing and developing 
the physical form and for final publication of a questionnaire. Emphasis is 
placed on consideration for the respondent, since the objective is to obtain as 
many responses as possible. Questionnaire forms that meet criteria of physical 
attractiveness and consideration, including materials, arrangement, directions, 
cover letter, envelopes, mailing, and followup technique, should receive the 
percentage of replies sufficient to fulfill the investigator's requirements. 


EVALUATION: Well-organized outline for questionnaire construction. 
KEYWORDS: Questionnaire: design. 


Norman, Ralph D. 
1948. A review of some problems related to the mail questionnaire tech- 
nique. Educ. & Psychol. Meas. 7(2): 235-247. 


A literature review of 37 references concentrated between 1930-48 shows 
that: followups increase the percentage of returns, rewards should be used with 
caution, the questionnaire format is important, the study sponsor's identity is 
important, best time to issue a questionnaire is early in the week and early in 
the school year, and too long a questionnaire may reduce returns. Respondents 
differ from nonrespondents in interest toward the questionnaire. Late respon- 
dents may differ from early respondents. 


EVALUATION: Good general summary supported by several tables illustrating 
comparisons. 
KEYWORDS: Literature review 

Bias: nonresponse 

Followup: mail 

Questionnaire: format 

Early vs. late response 

Incentive: reward, motivation. 
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Olson, Williard C. 

1936. The waiver of signature in personal reports. J. Appl. Psychol. 

20(4): 443-450. 

The Woodworth-Mathews Personal Data Sheet, a personality test to measure 
emotional instability, was given to two comparable groups of college women, 
one group remaining anonymous, the other group signing their names. The test 
was then given a second time with instructions reversed. In the initial applica- 
tion, subjects reported significantly more feelings and symptoms with neurotic 
implications under anonymous conditions than when required to sign their names. 
However, the initial test appeared to establish a set, or memory factor, pre- 
venting large changes on the second application to the same group. 
EVALUATION: Substantial evidence of questionnaire bias on very personal data 
with secondary evidence of memory restricting much change on the same ques- 
tionnaire. Well written but confusing presentation of quantitative data. 


KEYWORDS: Anonymous respondent. 


Oppenheim, A. N. 
1966. Questionnaire design and attitude measurement. 298 p., illus. 
New York: Basic Books. 


A comprehensive text on all phases of questionnaire use. Discusses prob- 


lems associated with the several approaches to survey and questionnaire research. 


Questionnaire construction chapters deal with question wording, attitudes and 
attitude scaling, use of projective techniques, and others, Analysis section 
deals with data coding, preparation of cards for computer analysis, and table 
analysis. Appendix lists established socioeconomic rankings for approximately 
500 occupational categories. 

EVALUATION: Excellent, comprehensive, up-to-date, readable references. 


KEYWORDS: Analysis 
Question wording 
Reliability 
Advantages and disadvantages of questionnaires 
Questionnaire: design 
Attitudes 
Textbook. 


Pace, Robert C. 
1939. Factors influencing questionnaire returns from former university 
students. J. Appl. Psychol. 23(3): 388-397. 


A 52-page questionnaire was sent to 1,507 former University of Minnesota 
students to determine characteristics and activities since leaving school. With 
69-percent return, the study results show that the method of comparing early 
versus late returns is not as sensitive to the true extent of bias as is the direct 
comparison of returns versus nonreturns, but this former comparison does 
indicate direction of bias. University graduation and number of years of school 
completed were important factors influencing returns, but sex, age, and year 
of entrance to the university were unimportant. 
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EVALUATION: Good study worth reading. 
KEYWORDS: Bias: nonresponse 
Early vs. late response. 


Parry, Hugh J., and Helen M. Crossley 
1950. Validity of responses to survey questions. Publ. Opin. Quart. 14: 
61-80. 


The article examines two concepts of validity (as a test to predict perfor- 
mance and as a matter of interpretation or definition), reviews the literature on 
the subject, and presents results of a specially designed survey in Denver 
which showed that the validity of even simple "‘factual'' responses may often be 
open to question. Invalidity was shown to commonly follow social pressures; 
with more respondents exaggerating their participation in elections than under- 
reporting it. The same tendency was noted in the report of possession of library 
cards and driver's licenses. 

EVALUATION: Excellent discussion on questionnaire validity both for philosoph- 
ical reasons and for practical reasons. 
KEYWORDS: Validity 

Literature review. 


Parten, Mildred 
1950. Surveys, polls, and samples: practical procedures. 624 p., illus. 
New York: Harper and Brothers. 


Presents information on procedures used by population surveyors and on 
evaluations of populations by questionnaires and related devices. Includes the 
historical background of population surveying and a description of significant 
current practices. Presented is a guide for planning the survey, drafting the 
forms, writing the instructions, securing the information, and interpreting and 
reporting the results. Particular emphasis is given to the problem of bias and 
to specific procedures which meet technical problems arising at each stage of 
the survey operation. Primary consideration is given to surveys that gather 
information about people rather than about the agencies or facilities provided 
for them. 

EVALUATION: Good reference text especially on historical aspects. Includes 
1, 145-reference bibliography. 
KEYWORDS: History 
Questionnaire: design 
Bias: nonresponse, representativeness, stated vs. actual behavior 
Analysis 
Textbook. 


Payne, Stanley L. 
1950a. Case study in question complexity. Publ. Opin. Quart. 13: 653-658. 
This article examines 16 questions given in a national survey of 6,400 people. 
Each of the questions contains an alternative question such as, ''Do you think that 
this tax is about right, or is it too high?"' Half the questionnaire had the alterna- 
tive in the question reversed such as, ''Do you think that this tax is about right, 
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127. 


or is it too low?" Nine of the questions proved to be "tight, '' in that differences 
in answers between the two forms did not exceed 2 percent. Seven questions, 
labeled as "loose, '' produced statistically significant different answers ranging 
from 4.5 percent to 7.5 percent. Results show that the loose question's alter- 
native was selected more often when it was presented last, opinions were held 
less strongly on the tight questions than on the loose ones, and question brevity 
and simplicity are important elements of "'tightness.'' Flesch Readability Scores 
revealed that the tight questions qualified as suitable for people who had com- 
pleted seventh or eighth grade, and the seven loose questions qualified as diffi- 
cult or suited for people of high school or more education. 


EVALUATION: This paper is addressed to interviewers but the generalities apply 
to mail surveys. 
KEYWORDS: Question: wording. 


1950b. Thoughts about meaningless questions. Publ. Opin. Quart. 14: 
687-696. 


Researchers should understand that respondents will answer questions that 
may be meaningless to them and that answers are not always random depending 
upon the respondents' predisposition. The proportion of respondents for whom 
questions are meaningless is impossible to determine and the fact that answers 
fit subsequent behavior does not prove that questions were meaningful. Even 
consistency of replies is no proof that a question is full of meaning. Precautions 
against meaningless questions include recognizing issues or terms that could be 
meaningless to many respondents, avoiding abstractions or words with vague 
and general definitions, avoiding long and complicated questions or words, using 
"filter" question to determine if respondent understands question or issue, using 
scaling techniques which will eliminate questions that do not "scale" with others 
on the same topic, and pretesting questions before they are used extensively. 


EVALUATION: Excellent suggestions, clear and well presented with examples 
to support contentions. 


KEYWORDS: Question: wording. 


1951. The art of asking questions. 249 p. Princeton, N.J.: Princeton 
Univ. Press. 


Major topics covered include the importance of asking good questions; a 
description of the open-end question and its demerits; a discussion of the two-way 
question and its duplicities; a discourse on the multiple-choice question and its 
misconstructions; a description of special types of questions and their faults; 
the concern for respondents' privacy; the virtues of brevity and simplicity; 
problems with individual words and questions; a short lesson on punctuation, 
phonetics, and abbreviations; a checklist of 100 considerations; and 53 references. 


EVALUATION: Clearly presented text on a subject deserving greater appreciation. 


KEYWORDS: Question: content, type, wording 
Textbook. 
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128. Pedersen, Darhl M., and Vincent J. Breglio 
1968. The correlation of two self-disclosure inventories with actual 
self-disclosure: a validity study. J. Psychol. 68: 291-298. 


Two self-disclosure measures and a self-disclosure questionnaire were 
administered to 52 subjects. Each of the self-disclosure measures yielded four 
scores which indicated the extent to which the subjects claimed they had disclosed 
information about themselves to four target persons (mother, father, best male 
friend, and best female friend). The questionnaire yielded separate scores of 
actual depth of self-disclosure in five topic areas (interest, personality, studies, 
body, and money), a total depth of disclosure score, and a total amount of dis- 
closure score. Correlations between the 10 scores and the seven questionnaire 
scores indicated that total depth of disclosure was highly correlated with total 
amount of disclosure; that both total depth and total amount of disclosure were 
correlated with mother and father; that the only area of actual disclosure con- 
sistently related to claimed disclosure was that of studies. 

EVALUATION: Worth reading if interested in overt versus verbal disclosed 
behavior. 
KEYWORDS: Bias: stated vs. actual behavior. 


129. Perry, Harold E. 
1925. The questionnaire method. J. Appl. Sociol. 10(Sept.-Dec.): 
155-158. 


Responses to two questionnaires completed by junior high teachers were 
rejected because the investigator "knew, "' from 2-year association with the 
subjects, that their true attitudes were not revealed. Reasons why the question- 
naire method is good for objective data but inaccurate for subjective data are 
given: (1) it is impersonal and fails to arouse emotions which reveal attitudes; 
(2) it allows variation in interpretation of questions; (3) questions are often 
answered to give certain impressions; (4) it encourages brief, concise answers 
which do not portray attitudes; (5) questions are often too direct and arouse 
antagonism or inhibition. 

EVALUATION: Dated, essay-style article based on intuition and vague experience 
rather than data. 
KEYWORDS: Advantages of questionnaires 

Disadvantages of questionnaires. 


130. Perry, Reginald 
1923. Putting persuasive power into a questionnaire. Printers' Ink 
Vol. 123, May 10, p. 125-129. 

Essay about using questionnaires to gather ideas for equipment advertising 
from superintendents of plants. Advises readers to leave plenty of white space 
between questions. Avoid "yes" or "no" questions since they will not provide 
new ideas. Write questionnaires in conversational style. Dovetail questions 
with one another to insure adequate information. 

EVALUATION: Dated, weak essay of no value for research purposes. 
KEYWORDS: Questionnaire: design. 


50 


131. 


132. 


133. 


Phillips, Marjorie 
1941. Problems of questionnaire investigation. Res. Quart. 12: 528-537. 
Seven disadvantages and three advantages of questionnaires are given. Nine 
items are given to consider before constructing a questionnaire, and 15 desirable 
practices are given for its construction. Ten questions are then offered as 
criteria for judging a questionnaire already constructed. A questionnaire is a 
cooperative effort between the individual constructing it and the respondents who 
should thus be treated with all possible consideration. Lengthy bibliography of 
questionnaire use in education included. 
EVALUATION: Good general advice but some of it outdated and specific to 
education. 
KEYWORDS: Disadvantages of questionnaires 
Advantages of questionnaires 
Questionnaire: design. 


Phillips, William M., Jr. 
1951. Weaknesses of the mail questionnaire: a methodological study. 
Sociol. & Soc. Res. 35(3): 260-267. 


To ascertain the extent of investigator control over representativeness of 
response, the author obtained data from a sample of 93 Fisk University graduates. 
By the use of followups, response was increased approximately 50 percent; by 
the use of personalizing devices (postage and personal letter), response was 
increased approximately 70 percent. No significant differences were found 
between early, late, and nonrespondents when the four factors of year of gradua- 
tion, sex, marital status, and number of children were tested. It was concluded 
that the investigator can control the representativeness of his sample by increas- 
ing response to a mailed questionnaire through use of followup and personalized 
devices. 

EVALUATION: Paper falls short of expectations aroused by title, but it does 
offer further evidence of the value of followups to questionnaire studies. 
KEYWORDS: Bias: nonresponse 

Followup: mail 

Farly vs. late response 

Incentive: motivation, postage. 


Plog, Stanley C. 
1963. Explanations for a high return rate on a mail questionnaire. Publ. 
Opin. Quart. 27: 297-298. 

The return rate for 162 questionnaires mailed to readers who had sent 
letters to the editor was 92 percent for one sample and 98 percent in a second. 
Factors affecting the return rate were respondent interest, mailing of question- 
naires the same day that letters were received, enclosure of individually typed 
letters of explanation and postage-free , addressed return envelopes. 
EVALUATION: Results have limited application and were based upon speculation. 
KEYWORDS: Incentive: motivation, postage 

Cover letter. 
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135. 


136. 
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Price, D. O. 
1950. On the use of stamped return envelopes with mail questionnaires. 
Am. Sociol. Rev. 15(5): 672-678. 


All 157 members of a regional sociological society were sent invitations 
to join a national sociological society. Half of the return envelopes were postage 
paid; the other half were not. After 6 months, 23.3 percent of those receiving 
stamped return envelopes and 17.3 percent of those receiving unstamped enve- 
lopes had joined. This significant difference in response rate was considered 
due to convenience since a $6 membership fee was required of all who joined. 
EVALUATION: Brief note indicating the value of return postage. 
KEYWORDS: Incentive: postage. 


Reid, Seerley 
1942. Respondents and nonrespondents to mail questionnaires. Educ. 
Res. Bull. 21(4): 87-96. 


School principals (3,293) were contacted by questionnaire to determine 
the extent of audio-visual equipment being used, and 42 percent responded 
initially. Special delivery letters and telephone calls to 92 names randomly 
selected from the remaining (1,032) nonrespondents increased response 69 
percent. The replies of the 87 that answered were assumed to be representa- 
tive of the nonrespondents. On nine out of the 10 questions, there were 
statistically significant differences between respondents and nonrespondents. 
Differences reflected less interest in topic by nonrespondents and indicate 
that respondents are not representative of nonrespondents. Every questionnaire 
study should include a followup. Nonrespondents can be sampled rather than 
resurveyed in total. Review of previous studies of nonresponse bias. 


EVALUATION: Good example of nonresponse bias. Extent of bias may have 
been greater with some other topic. 


KEYWORDS: Bias: nonresponse 
Followup: mail, telephone. 


Reuss, Carl F. 
1943. Differences between persons responding and not responding to a 
mailed questionnaire. Am. Sociol. Rev. 8(4): 433-438. 


Differences between student respondents and nonrespondents to a mail 
questionnaire were analyzed for differences in (1) intelligence, (2) length of 
stay in college, (3) community backgrounds, (4) family backgrounds, and (5) 
social participation. Intelligence and amount of college were positively related 
to response. Students from rural backgrounds responded more than those with 
urban backgrounds. Farm children responded better than children of skilled 
laborers. Sorority and fraternity members responded more than nonmembers. 
Initiative and strength of ties to the school (interest) generally increased returns. 
EVALUATION: Dated article that supports differences between nonrespondents 
and respondents. 
KEYWORDS: Bias: nonresponse. 


137. 


138. 


139. 


Robin, Stanley S. 
1965. A procedure for securing returns to mail questionnaires. Sociol. 
& Soe. Res. 50(1): 24-35. 


A literature review indicates differences between respondents and non- 
respondents when the study is concerned with opinions, values, and psychological 
characteristics but no difference when the questionnaire deals with preference 
and behavior. Incentives such as money or registered letters rarely increase 
returns to interview levels. Author suggests a maximum of five contacts with 
respondents: a prequestionnaire letter, a questionnaire with cover letter, a 
followup letter, a second questionnaire, and a third followup letter. Content 
of cover letter is suggested along with a mailing interval of 7 days. Results 
from nine questionnaire studies show no basis for concluding that prequestion- 
naire letters made significant differences in returns. 

EVALUATION: Excellent literature review and discussion of questionnaire 
returns. Interpret with caution the nine studies because eight deal with student 
respondents who may not be representative of other respondents, 
KEYWORDS: Cover letter 

Bias: nonresponse 

Followup: mail 

Prequestionnaire letter 

Literature review 

Incentive: money. 


Robinson, R. A. 
1952a. How to design a mail survey. Printers' Ink 239(9): 27-30. 


Planning and construction of the mail survey include: (1) obtain background 
on the subject; (2) prepare a preliminary list of questions; (3) design the sample 
to be representative of the population; (4) use accurate, complete, and adequate 
lists or files, which are current and not subject to duplication; (5) pretest the 
questionnaire by either direct mail to the respondents or by personal inter- 
views; (6) carefully check the questionnaire before mailing to avoid coding and 
tabulating bottlenecks; and (7) send along a collated sample of all enclosures 
to insure proper mailing. 

EVALUATION: Brief cookbook outline of questionnaire construction. Excellent 
for the beginner. 
KEYWORDS: Questionnaire: design. 


1952b. How to boost returns from mail surveys. Printers’ Ink 258(June): 
35-37. 


A variety of procedures designed to increase the probability of question- 
naire returns from mail surveys are briefly discussed. The author points out 
that response rates can be increased by properly designing the questionnaire 
itself; providing monetary inducement for its completion; including a warm, 
friendly letter of introduction; providing a stamped, return envelope; and 
sending several followup letters and at least one additional questionnaire to 
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140. 


141, 
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nonrespondents. The author points out that, with proper precautions, response 
to mail questionnaire studies can be increased appreciably above what would 
otherwise be expected. Data are presented which demonstrate that nonresponse 
bias is unimportant when the return rate exceeds 80 percent. 


EVALUATION: Although much has been learned about mail questionnaires since 
this paper was written, the major problems associated with constructing ques- 
tionnaires and prompting their return are well summarized. 
KEYWORDS: Questionnaire: design 

Incentives: money, motivation, postage 

Followup: mail 

Cover letter. 


, and Philip Agisim 
1951. Making mail surveys more reliable. J. Marketing 15: 48-56. 


A men's wear survey of 5,960 subscribers to a leading national magazine 
showed a 93-percent return after a 25-cent inducement was given in the original 
questionnaire and a $1 tie clasp was offered in a followup reminder. No signifi- 
cant difference was found between the responses before and after the followup. 
Authors conclude that when returns reach 80 percent, reliability of data is 
little affected by nonrespondents. In the followup, a question was included to 
determine why people had not responded. Almost half said they mislaid it or 
overlooked answering, and only 7 percent refused outright. This finding indi- 
cated the importance of followup mailings with an additional questionnaire 
included. The authors examined several studies that had high return rates to 
determine factors that may be helpful in conducting questionnaire studies. 
These include: preliminary study of the survey population; pretesting question- 
naires to discover bias or ambiguities; careful design and format of question- 
naire; inclusion of some sort of incentive for respondents; inclusion of a cover 
letter; inclusion of a return envelope preaddressed and stamped; and the use of 
followup reminders. In nine studies, the daily returns showed that 50 to 60 
percent of the total returns come within 7 days, 90 percent after 2 weeks, and 
nearly all questionnaires should be received after 3 weeks. 

EVALUATION: Excellent paper that corresponds to the objectives behind this 
annotated bibliography. Three references. 
KEYWORDS: Bias: nonresponse 

Followup: mail 

Cover letters 

Pretest 

Questionnaire: design, format 

Incentive: reward 

Early vs. late response. 


Roeher, G. Allan 
1963. Effective techniques in increasing response to mailed question- 
naires. Publ. Opin. Quart. 27: 299-302. 
A 32-item attitude and information scale was sent to 400 subjects living 
in both rural and urban areas who had donated to Easter seals and the March of 


142. 


143, 


Dimes. One-fourth of the cover letters gave no title to the endorser and the 
remainder had a fictitious title, ''Director of Rehabilitation, "' after the name. 
Of those who received questionnaires with a title, 81 percent returned them; 

of those without a title, 55 percent. Also, urban residents responded better 

(83 percent) than rural subjects (72.3 percent). 

EVALUATION: Findings are suggestive but lack rigorous analysis to determine 
significance and amount of variation due to the various factors. Lack of evalua- 
tion of nonresponse further weakens results. 12 references, 


KEYWORDS: Cover letter 
Bias: nonresponse, 


Rollins, Malcolm 
1940. The practical use of repeated questionnaire waves. J. Appl. 
Psychol. 24(6): 770-772. 


In.a study by Cosmopolitan Magazine to determine for advertising pur- 
poses whether their readers traveled by commercial airlines, 750 questionnaires 
were sent to subscribers. Initial returns were less than 23 percent, and 13 
percent of a followup wave were returned. Seventeen percent of the original 
respondents had flown compared with only 7 percent of the followup respondents, 
Respondents under age 55 were interested in lower cost, and respondents over 
45 interested more in safety. 

EVALUATION: Interesting example of how nonresponse bias can affect results. 
KEYWORDS: Bias: nonresponse 

Followup: mail 

Incentive: motivation. 


Rosen, Hjalmar, and R. A. Hudson Rosen 
1955. The validity of "undecided" answers in questionnaire responses. 
J. Appl. Psychol. 39(3): 178-181, illus. 


A questionnaire to union members was supplemented by interviews with 

a sample of nonrespondents to determine whether responses in the "undecided" 
category can be considered as valid evidence that individuals have not made 
definite judgments. Opinion items consisted of statements on norms, perceptions, 
and evaluations. Findings for the undecided group, that indicated definite per- 
ceptions, cast considerable doubt on the validity of middle category (undecided, 
uncertain) responses. In no case was the undecided group both significantly 
less homogeneous than the satisfied group and significantly more homogeneous 
than the dissatisfied group. The "undecided" judgments of members who did 
not know what was being done on any point would appear to be valid. There is 
no evidence for the validity of other undecided responses. It may be advantageous 
to omit the undecided category. Respondents undecided because of insufficient 
information could leave the question blank, but others may be lured into taking 
a stand, : 
EVALUATION: Excellent paper worth considering when designing questions 
containing "undecided, "' "don't know, "' 'no comment," or "neutral. "' 
KEYWORDS: Validity 

Interpretation 

Question: wording. 
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145. 
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Rosen, Ned A. 
1960. Anonymity and attitude measurement. Publ. Opin. Quart. 24: 
675-679. 

In a preliminary investigation of the effectiveness of the developmental 
reading program at Purdue University, 678 first-semester college freshmen 
completed two questionnaires. Half of the students were requested to sign their 
names and half were not. Identification of respondents in attitude questionnaire 
surveys conducted under less than highly threatening circumstances is not likely 
to result in serious statistical or practical distortion. This position is supported 
by eight of 10 studies cited. Where respondent identification is essential for 
correlational or followup purposes, the straightforward approach is preferable 


‘to a number coding system. In sensitive issues or where there is expected 


distortion, it may be advisable to use an anonymous questionnaire. 
EVALUATION: Caution is advised in extending results of this study to studies 
involving other age groups who may react differently to signing their names. 
KEYWORDS: Anonymous respondent 

Questionnaire: design. 


Rosenau, James N. 
1964. Meticulousness as a factor in the response to mail questionnaires, 
Publ. Opin. Quart. 28: 312-314. 


A short questionnaire asking for name, home address, organizational 
affiliation, phone number, and availability for volunteer work was given to 
1,067 national leaders attending a White House conference on foreign aid. Only 
32.4 percent were returned. It was hypothesized that idiosyncratic differences 
rather than interest or involvement in conference topic distinguished respon- 
dents and nonrespondents, and that those who returned the questionnaire would 
more likely respond to a mail questionnaire. A second questionnaire was mailed 
to the same 1,067 conferees after they returned home, Of the 61 percent who 
returned questionnaires, a significant 75.1 percent of those who returned the 
first questionnaire responded. Of those who did not respond to the first question- 
naire, 51.6 percent responded to the second one. Cross-tabulation revealed 
that interest and involvement in the topic was not associated with rate of return. 
Author concludes that response to surveys is a function of the respondent's habits 
and attitudes that are highly personal and deeply ingrained. 
EVALUATION: Short but interesting paper pointing to an inherent weakness in 
the use of mail questionnaire. 
KEYWORDS: Bias: nonresponse 

Incentive: motivation. 


Roslow, Sydney, and Albert B. Blankenship 
1939. Phrasing the question in consumer research. J. Appl. Psychol. 
23(5): 612-622. 


Included in nine principles considered in the phrasing of questionnaire 
items are rapport, nonambiguous and concrete questions, question's position in 
the schedule, importance of respondents' background, nonemotional and unbiased 
questions, and possible influence of multiple choice-type questions. 
EVALUATION: Clear statement of sound principles, still timely. 

KEYWORDS: Question: wording. 


147. 


148. 


149, 


150. 


Ruckmick, Christian A. 
1930. The uses and abuses of the questionnaire procedure. J. Appl. 
Psychol. 14(1): 32-41. 


Five weaknesses of the questionnaire are given: (1) It stresses explicit 
categorical answers, (2) it does not account for individual reactions, (3) its 
results may depend upon the researcher's interpretation, (4) its statistical 
treatment is often misleading, and (5) nonresponse and the percentage of replies 
left blank can seriously affect findings. Three advantages of questionnaires 
are given: (1) large amounts of data can be gathered in a short time, (2) insights 
can be gained by its exploratory approach to problems, and (3) whoever con- 
structs a questionnaire is forced to define his problem and the implications 
at the onset. 


EVALUATION: Advances during last 40 years make disadvantages seem less 
serious and advantages obvious. 
KEYWORDS: Disadvantages of questionnaires 

Advantages of questionnaires. 


Rugg, Donald 
1941, Experiments in wording questions: II. Publ. Opin. Quart. 5: 91-92. 
A simple experiment on question wording indicates that alternate word 
forms may result in different response patterns among certain socioeconomic 
groups. 
EVALUATION: The findings are not definitive, however; and no recommenda- 
tions are made by the author. 
KEYWORDS: Question: wording. 


Rushmore, Elsie M. 


1934. How to get results in mail questionnaires. Printers' Ink 166(March): 


17-21. 

As a basis for evaluating questionnaire construction, 24 questions are 
given; and 14 additional questions are given to evaluate reports based upon the 
findings of questionnaire studies. States that questionnaires are more reliable 
for quantitative factual data and interviews more reliable for subjective personal 
data. Weather favors questionnaires over interviews since poor weather makes 
standing on the doorstep impractical, but good weather decreases the number of 
people at home. 

EVALUATION: Logical, general guidelines reflecting the era in which they 
were written. 
KEYWORDS: Questionnaire: design 

Mail questionnaire vs. interview. 


Russell, Susan D., Robert Konrad, and Arnold D. Kaluzny 
1970. Influencing the respondent: an experiment in maximizing the res- 
ponse rate of mail questionnaires. Sociol. Abstr. 18(3): 22. 


Mail questionnaires were sent to 623 hospital administrators to test the 
effect of three mailing conditions: postage--regular, airmail, or airmail-special 
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delivery; presence or absence of postscript; and presence or absence of "personal" 
stamped on the envelope. None of the variables had a direct effect, but a com- 
bination of airmail-special delivery, a postscript, and "personal" marked on 

the envelope brought in a 92-percent return. Only 67-percent return was rea- 
lized from the combination of airmail, no postscript, and no "personal" stamped 
on the envelope. The highest response was from the closest area from question- 
naire origin. The organizational size was related to response rate with the 
smallest hospitals having the lowest return rates. 

EVALUATION: The article is a very brief description of a study with no dis- 
cussion of findings. 

KEYWORDS: Incentive: postage, motivation. 


151. Salisbury, Philip 
1938. 18 elements of danger in making mail surveys. Sales Manage. 
42(Feb. ): 28-30, 84-85. 


Chances for misinformation and misinterpretation are greater for mail 
surveys than for interviews. Handicaps of mail surveys include: they take too 
much of respondent's time; mailing lists are inaccurate; surveys take too long 
to complete and are more costly than assumed; certain answers are inaccurate 
when written; nonresponse is high; representativeness is questionable; person 
sampled may not be the one who answered questions; response is highest for 
those most interested; negative opinion holders are most likely to respond; 
incomplete questionnaire is more likely; income is likely to be overstated for 
prestigious reasons; there is no chance to qualify unclear questions and no 
chance to evaluate subjectively respondent's mood. 

EVALUATION: Handicaps largely overcome through more refined questionnaire 
design and sampling techniques. Many handicaps listed were poorly supported. 
KEYWORDS: Bias: nonresponse 

Disadvantages of questionnaires 

Mail questionnaire vs. interview. 


152. Schwirian, Kent P., and Harry R. Blaine 
1966. Questionnaire return bias in the study of blue-collar workers. 
Publ. Opin. Quart. 30: 651-653. 


Analysis of questionnaire returns from the study of members of the 
United Automobile Workers Union gave significant information on the nature 
and direction of questionnaire return bias. Three hypotheses were explored: 
(1) that questionnaire return bias is characteristic of blue-collar populations, 
as of other populations; (2) that the direction of the bias is predictable; and 
(3) that the nature of the associations among the variables is not disturbed by 
the bias. Initial respondents (26.5 percent) to a mailed questionnaire were 
examined separately from those who responded to a second mailing (27.0 percent). 
The data lent support for the first two hypotheses. For the third, however, 
contradictory evidence was found. Not only does return bias have a serious 
effect upon the estimation of population parameters, but the findings suggest 
that the association between two variables may be affected. 

EVALUATION: The authors properly conclude that researchers not only must 
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153. 


154. 


155. 


strive to obtain the maximum possible returns to mailed questionnaires but must 
be sensitive to the subtle effects of return bias. 
KEYWORDS: Bias: nonresponse. 


Scott, Christopher 
1961. Research on mail surveys. J. Royal Statist. Soc. 124(2): 143-205. 


Five mail surveys by the British Government are examined along with the 
literature on mail questionnaires. Experimental features introduced into these 
studies allow measurement of nonresponse bias, early versus late questionnaire 
returns, response by nonaddressees, the influence of a variety of factors on 
the response rate, and response reliability and validity. An attempt is also 
made to evaluate all published research on each topic. 120 references. 


EVALUATION: Excellent state-of-the-art compilation. 
KEYWORDS: Bias: nonresponse, representativeness 

Early vs. late response 

Literature review 

Followup: mail 

Questionnaire: length 

Cover letter 

Incentive: postage. 


Scott, Frances Gillespie 
1957. Mail questionnaires used in a study of older women. Sociol. & 
Soc. Res. 41: 281-284, 


To ascertain the relationship between personal adjustment in old age and 

occupational mobility, the author sent questionnaires to 360 women, 60 years 
of age and older, randomly selected from poll-tax exemption lists. The sample 
was divided into four parts to test the usefulness of mailing time and the use 
of a preliminary letter explaining the study. Earlier responses were received 
when the questionnaire was delivered early in the week, but the ultimate res- 
ponse rate after followup was significantly higher for questionnaires mailed later 
in the week. A chi-square test revealed no difference in ultimate responses 
between those receiving and those not receiving a preliminary letter. The 
preliminary letter may have increased interest and caused respondents to reply 
promptly. Apparently the followup letter had a similar effect on those who did 
not receive the preliminary letter. 
EVALUATION: Excellent but brief paper on a topic generally underestimated-- 
timed mailing. 
KEYWORDS: Followup: mail 

Prequestionnaire letter 

Incentive: postage. 


Seitz, Richard M. 
1944, How mail surveys may be made to pay. Printers' Ink 209(Dec.): 
17-19, 96, 98, 99, 102, 


Advantages of direct mail questionnaires are enumerated as well as their 
disadvantages. Primarily directed to readers wishing to use questionnaires 
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commercially. 'Non-white collar groups are not regarded as good possibilities 
since they are often unaccustomed to writing. '' Suggestions on form, wording, 
cover letter are made as well as ways to increase returns. 
EVALUATION: Casual, offhand advice, much of which is misleading; methodology 
is unsound. 
KEYWORDS: Questionnaire: design 

Advantages of questionnaires. 


156. Selltiz, Claire, Marie Johoda, Morton Deutsch, and Stuart W. Cook 
1959. Research methods in social relations. 622 p. New York: Holt 
Rinehart & Winston. 


Chapters 7 and 10 and Appendix C pertain to mail questionnaires. Dis- 
cussed are problems of validity, questionnaires versus interviews, question 
content, type of questionnaires, and scaling. Questionnaire construction 
includes: (1) deciding what information is needed, (2) deciding on the type of 
questionnaire and writing the first draft, (3) reexamining and revising questions, 
(4) pretesting, (5) editing questionnaire, and (6) specifying procedures for its 
use. 


EVALUATION: Systematic and thorough discussion of the basic principles of 
questionnaire construction and use. Worth consulting. 
KEYWORDS: Questionnaire: design 

Mail questionnaire vs. interview 

Advantages of questionnaires 

Scaling 

Disadvantages of questionnaires 

Question: content 

Pretest 

Textbook. 


157. Shafer, Elwood L., Jr., and John F. Hamilton, Jr. 
1967. A comparison of four survey techniques used in outdoor recreation 
research. USDA Forest Serv. Res. Pap. NE-86, 22 p., illus. 
Northeast. Forest Exp. Stn., Upper Darby, Pa. 


Handout questionnaires and mailed surveys--with three to four waves for 
nonrespondents--provided results that were comparable to personal interview 
data in 75 to 87 percent of the question items. Delayed mail surveys conducted 
3 months after the camping experience,with questions of the same type or of 
similar complexity as those in the present study, are recommended over handout 
or immediately mailed surveys. 
EVALUATION: Good example of various techniques used in questionnaire research. 
KEYWORDS: Advantages of questionnaires 

Mail questionnaire vs. interview. 


158. Shannon, J. R. 
1948. Percentages of returns of questionnaires in reputable educational 
research. J. Educ. Res. 42(2): 138-141. 


A total of 639 research studies in master's theses, Ph.D. dissertations, 
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159. 


160. 


161. 


and articles in the Journal of Educattonal Research using questionnaires were 
examined to determine their rate of returns. The mean percentage of returns 
for the 433 usable studies was 72 percent, but 206 studies lacked enough infor- 
mation to compute returns. 
EVALUATION: Article is very generalized but does point out inadequacies in 
research report writing. 
KEYWORDS: Bias: nonresponse 

Literature review. 


Sharp, Harry 
1955. The mail questionnaire as a supplement to the personal interview. 
Am. Sociol. Rev. 20(6): 718. 


Survey was made of demographic and economic information on 2, 556 adults 
in a Detroit area survey. Questionnaires were left at each household where any 
adult contacted could not supply the information needed on adults who were not 
present. The mail questionnaires were used only to a limited extent, but 40 
percent or 21 of the 53 respondents for whom questionnaires were left did return 
the forms. The use of the mail questionnaire as a supplement to the personal 
interview did result in a more complete collection of data with comparatively 
little additional effort on the part of the researchers. 

EVALUATION: Good example of the use of questionnaires to supplement an 
interview survey. Nonresponse was extremely high which reduced the reliability 
of the questionnaire portion of the study. 
KEYWORDS: Applications 

Mail questionnaire vs. interview. 


Shuttleworth, Frank K. 
1931. A study of questionnaire technique. J. Educ. Psychol. 22(Dec.): 
652-658. 


A six-item questionnaire evaluating the effectiveness of a rural public 
health project was sent to 608 persons with a 25-cent coin enclosed, and to 376 
persons with no coin enclosed. There was 52-percent response from those who 
received the coin but only 19-percent response from the no-coin sample. Coin 
questionnaires were 12 percent more expensive in terms of unit cost per return. 
EVALUATION: The value of money incentive is illustrated, but serious nonre- 
sponse may still existin the 52-percent return. Extensive followup reminders 
inducing higher returns may be cheaper. 

KEYWORDS: Incentive: money. 


1941. Sampling errors involved in incomplete returns to mail question- 
naires, J. Appl. Psychol. 25(5): 588-591. 


To test sampling errors due to incomplete questionnaire returns, great 
effort was invested in securing complete returns in a survey of the employment 
status of 327 chemistry graduates. Results after 60 percent (184 questionnaires) 
were returned showed 0.5 percent were unemployed. After another 125 question- 
naires were received, the unemployment rate was 5.6 percent; but after 
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100-percent return, the unemployment rate stood at 4.0 percent. Suggests 
securing complete returns from at least a portion of the total population in a 
pretest to aid planning to help minimize sampling errors. 
EVALUATION: Concise article demonstrating possible bias resulting from non- 
response to mail questionnaires. 
KEYWORDS: Bias: nonresponse 

Pretest. 


Simon, Raymond 
1967. Responses to personal and form letters in mail surveys. J. Adver. 
Res. 7(1): 28-30. 


Two readership studies of industrial company magazines and a study of 
subscribers' attitudes toward a hospital insurance plan provided an opportunity 
to compare the effects of a personally typed cover letter and a mimeographed 
form letter. In all three studies, 2,414 questionnaires were sent out to employee 
readers and general public readers. Questionnaire returns ranged from 26 per- 
cent to 60 percent. Results indicate that personally typed letters have no clear- 
cut advantage over mimeographed form letters in terms of percent response. 
EVALUATION: The author's opening statement that "a form letter produced 
greater returns than a personal letter'' is contrary to his results. The nonre- 
sponse level inall three studies is extremely high. Eight references. 
KEYWORDS: Cover letter. 


Sjoberg, Gideon 
1954. A questionnaire on questionnaires. Publ. Opin. Quart. 18: 423-427. 


Two census tracts in Texas were surveyed to determine what attitudes 
people have toward questionnaires and interviewing. Both tracts were predom- 
inantly "Anglo-American, '' but one had high income and education; the other had 
low income and education. More people in tract 1 (72 percent) had heard of the 
Gallup, Belden, and Roper polls than had persons in tract 2 (17 percent). Eighty- 
three percent of the respondents in tract 1 and 70 percent in tract 2 believed that 
people should be asked their opinions about political and social problems. Most 
people were opposed to having the city exercise restraint on interviewing. How- 
ever, only 52 percent in tract 1 and 38 percent in tract 2 favored having inter- 
viewers come to their homes. Little more than half the people preferred personal 
interviews to mailed questionnaires or telephone contacts. Interviewees were 
more willing to answer questions concerning religious beliefs and their jobs than 
to answer questions about money matters, political beliefs, and family life. In 
tract 1, 78 percent, and in tract 2, 55 percent responded with ''Yes"' when asked, 
"Do you believe most people answer honestly the questions they are asked by the 
interviewers ?"' 

EVALUATION: Interesting findings on type of questions respondents like and 
dislike most. Survey oriented toward interviews but has application for mail 
questionnaire surveys. 

KEYWORDS: Mail questionnaire vs. interview. 
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Sletto, Raymond F. 
1940. Pretesting of questionnaires. Am. Sociol. Rev. 5(Apr.): 193-200. 


Over 6 months were spent pretesting a questionnaire to be sent to 1, 600 
former university students. Length of questionnaire did not significantly affect 
the response rate of a 10-, 25-, and 35-page questionnaire in a 300-member 
pretest group. Three different cover letters did not significantly affect the rate 
of return. One cover letter requested help on an altruistic basis, another on a 
challenge basis, and a third on a "help us" basis. To test post-card versus 
letter followup notices, half of the nonrespondents from above received the 
post card and the other half, the letter. The number of returns was identical 
for each method. A third test involved recipients of the 25-page questionnaire 
who were asked to check difficult, dull, or unrelated sections of the question- 
naire. Twenty-two of the 60 respondents reacted unfavorably to questions 
relating to problems in the communities. These questions were rearranged 
with other, more favorable questions so their impact would be reduced. Another 
test asked 47 students to rate the esthetic appearance of 10 different page for- 
mats. Results indicate that esthetic preferences are not highly individualistic 
nor erratic. The final questionnaire containing 52 pages yielded 69-percent 
return. Author concludes that greater attention should be directed at pretesting 
to discover the selective factors that will bias the final questionnaire. 
EVALUATION: Excellent study demonstrating the importance of pretesting. 
Well worth reading before starting a questionnaire study. 

KEYWORDS: Cover letter 
Pretest 
Followup: mail 
Question: type 
Questionnaire: format, length. 


Slocum, W. L., L. T. Empey, and H.S. Swanson 
1956. Increasing response to questionnaires and structured interviews. 
Am. Sociol. Rev. 21(2): 221-225. 


Results of four studies involving farmer-hunter relationships, occupational 
planning of high school seniors, consumer habits, and college student withdrawals 
show that efforts to establish an image of the social utility of a survey and to 
emphasize the special role of each respondent will maximize response. Impro- 
vising and designing approaches to fit specific situations are necessary for 
utilizing these principles. 

EVALUATION: High response case studies used to support hypothesis. Poor 
control over other variables that probably contributed to response rate. 
KEYWORDS: Incentive: motivation. 


Smith, Francis F. 
1935. The direct validation of questionnaire data. Educ. Admin. & 
Superv. 21(8): 561-575. 


In an analytical inquiry into the validity of questionnaire data, various studies 
showed that individual response, taken by itself, is questionable. Individuals 
differ in their ability to furnish valid data such as estimates of size, number, 
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time, speed, and so on, particularly under rigorous standards. Validity is 
increased by increasing the sample size and merging their responses. In no 
case studied does the validity of questionnaire data seem to meet the rigorous 
demands made by more precise sciences. The validity of responses involving 
judgment and opinion is usually lower than for responses involving factual data. 
But factual data involving personal accomplishment often has prestige bias. 
EVALUATION: Dated but good basic data presented with analytical support. 
KEYWORDS: Validity 

Literature review. 


Smith, Mapheus 
1933. A note on stability in questionnaire response. Am. J. Sociol. 
38(5): 713-720. 


Fifty college students were given a 60-item questionnaire dealing with 
factual family items, factual personal items, and subjective personal items. 
Four months later they retook the questionnaire, and 780 answers were different. 
Average change per person on 16 family items was 4.0 questions for 22 males, 
and 3.1 for 28 females. Changes in 19 factual personal items were 4.3 for males 
and 3.5 for females. Changes in 25 subjective personal items were 10.0 for 
males and 6.9 for females. Forty-six of the students changed 10 or more items 
each, 

EVALUATION: Many items reported may have logically changed in the inter- 
vening 4 months of study. For example, ''Are you engaged?" ''Age of mother, ' 
or ''Do you dream often?" 

KEYWORDS: Reliability. 


- 


Stanton, Frank 
1939. Notes on the validity of mail questionnaire returns. J. Appl. 
Psychol. 23(1): 95-104. 


A three-page mail questionnaire concerning the use of classroom radio 
facilities was sent to 11,169 U.S. schoolteachers, and 28.3 percent were re- 
turned. Results showed that the first half of the respondents' answers did not 
differ significantly from the last half, and the first 95 percent to respond did 
not differ from the last 5 percent. After 5 days, 43.5 percent of the total re- 
sponses had been received. Another sample of 4,942 teachers was sent the 
same questionnaire; however, a followup questionnaire was sent on the 17th day 
after the original mailing. This time, 50.2 percent responded, and results 
showed a much higher proportion of teachers who did not use the radio facilities. 
These two examples indicate that the use of followup technique produces a signifi- 
cant increase in returns and gives a more representative sample in that people 
who were less interested or believed they had little to contribute were stimulated 
into responding. 

EVALUATION: This is one of the few actual tests of the effects of followup to 
stimulate response. In addition, the author claims to have received a 94-percent 
return using three followup contacts in another study. 
KEYWORDS: Followup 

Bias: nonresponse 

Early vs. late response. 
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Stephan, Frederick F. 
1948. History of the uses of modern sampling procedures. Am. Statist. 
Assoc. J. 43(241): 12-39. 


An extensive history of sampling is presented. Empirical knowledge is 


derived from incomplete observation and is a sampling of experience. Principles 


of sampling emerged from common sense, custom, and experience arising from 
needs of agriculture crop estimates, economic Statistics, statistical phases of 
social and health surveys, and public opinion polling. Probability theory, though 
well established in the 18th century, was not applied until the 20th with statistical 
needs of the government increasing during the depression. Complex sampling 
systems have been designed to fit the sample population, the costs and adminis- 
trative factors, and the principles of efficient design. Sampling choices are 
based on the study purpose, knowledge of the material, and experience with 
previous studies. Most current opinion polls and market research studies 
employ quota methods. The second war has been a greater accelerating stimulus 
to sampling than the depression of the 1930's. 
EVALUATION: Excellent historical perspective of survey research in general. 
Questionnaire research as such is not stressed. 
KEYWORDS: Literature review 

History 

Applications, 


Stoke, Stuart M., and Harvey C. Lehman 
1930. The influence of self-interest upon questionnaire replies. Sch. & 
Soc. 32(822): 435-438. 


Studies of individual members of seven classes on educational psychology 
lead to one conclusion: It is naive to assume that the lazy student would give 
an accurate report of his own laziness. Probably little reliance can be placed 
on students' statements of how much they study. 

EVALUATION: Dated, but points to problem of soliciting self-evaluation on 
sensitive subject. 
KEYWORDS: Reliability 

Incentive: motivation 

Bias: stated vs. actual behavior. 


Suchman, Edward A, 
1962. An analysis of "bias" in survey research. Publ. Opin. Quart. 26: 
102-111. 


Given the fact that nonresponse, does introduce a bias, an important ques- 
tion is not only ''Are the data biased?" but also ''How does the presence of bias 
affect the relationship between variables ?' Despite a biased frequency distribu- 
tion of interest and education in a panel study, the interrelationship between 
education and interest was not affected by this biased sampling. Another panel 
study found that even though educated people respond more, they did so regard- 
less of whether they were Catholic or Protestant. The presence of this bias 
did not affect the relationship between education and religion. The differential 
effect of bias appears to be a key factor in evaluating the importance of bias. 
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If a bias is constant (e.g., all educated people respond more regardless of 
interest, and vice versa), then the biased and unbiased samples will show the 
same results. However, if the bias is differential (e.g., the uninterested 
educated people and the interested uneducated people respond more), then the 
respondent and nonrespondent may differ. Finally, when the emphasis is upon 
the phenomenon being studied and not upon its distribution in the general popula- 
tion, bias may be used intentionally to one's advantage. 
EVALUATION: Good perspective of bias with a rigorous approach to it. 
KEYWORDS: Bias, nonresponse 

Validity. 


and Louis Guttman 
1947. A solution to the problem of question "bias."’ Publ. Opin. Quart. 
11: 445-455, illus. 


Presents an objective method of dividing respondents into pro and con 
groups which are relatively independent of question wording. The procedure 
for eliminating question "'bias'' consists of: (1) asking many questions on the 
same topic; (2) determining by scale analysis whether questions ask the respon- 
dent about the same dimension of opinion; (3) asking "How strongly do you feel 
about this?" after each opinion question; (4) relating content of opinion to intensity 
of feeling. This determines the zero or neutral point for content. The theory 
that the same intensity curve will be obtained by any sample of questions from 
the same scale can be tested by taking two series of opinion questions on the 
same topic which are biased in opposite directions and determining if the 
intensity measurement serves to correct these biases. Two examples illustrate 
this study. 

EVALUATION: Excellent paper backed with objectivity. A must for those 
constructing scale-type questions. 
KEYWORDS: Questionnaire: design 

Question: wording 

Sealing. 


and Boyd McCandless 
1940. Who answers questionnaires? J. Appl. Psychol. 24(6): 758-769. 
Two studies involving radio listeners each received response rates of over 
95 percent. In the first study, original mail questionnaires were followed by 
a second followup, and finally by a telephone contact of nonrespondents. In the 
second study, the original questionnaire was followed by three followup question- 
naires with the last questionnaire simplified and shortened to contain only key 
questions. Two main factors were found to influence the returns: interest or 
familiarity with the topic under investigation--the more interest, the greater the 
returns; and education of the respondent--the better educated, the greater the 
returns. For the analysis, respondents of the original questionnaire were com- 
pared with those responding to followup questionnaires. A biasing effect would 
have occurred if the researchers had not sent followup questionnaires. 
EVALUATION: Good example of the value of followup contacts with study subjects. 
KEYWORDS: Bias: nonresponse 
Early vs. late response 
Followup: mail, telephone. 


174, Tallent, Norman 
1959. A note on an unusually high rate of returns for a mail questionnaire. 
Publ. Opin. Quart. 23: 579-581, illus. 


A total of 1,567 mimeographed questionnaires with a covering letter of 
explanation were mailed to 393 psychologists, 741 psychiatrists, and 433 social 
workers, all employees of the Veterans Administration, as part of a study on 
the preparation of clinical reports. Completed forms were returned by 384 psy- 
chologists (97.7 percent), 602 psychiatrists (81.2 percent), and 421 social workers 
(97.2 percent). Three solicitations for cooperation were made: the initial 
questionnaire and a first and second followup letter. A personal cover letter 
and a postage-free return envelope also increased response. The questionnaire 
form had brief directions, a simple method of indicating choice of response, 
and space for comment. Establishment of a deadline and the enclosure of a 
duplicate questionnaire after the second followup letter also may have increased 
returns. 

EVALUATION: Return rate most likely due to professional commitment; study 
lacks rigorous analysis. 
KEYWORDS: Cover letter 

Questionnaire: design 

Bias: nonresponse 

Incentive: motivation, postage. 


175. Toops, Herbert A. 
1923. Validating the questionnaire method. J. Personnel Res. 2(4&5): 
153-169, illus. 


Five followup letters gave 93-percent returns from 93 trade school grad- 
uates. Graphs show number and timing of returns resulting from each followup. 
The cost per return decreased as the number of returns increased. Question- 
naires arriving at the recipients' homes on Friday and Saturday were answered 
more often than ones arriving at the beginning of the week. Plotting a cumula- 
tive curve of daily returns can indicate when followup letters should be sent. 
Copies of all followup letters included. 

EVALUATION: An excellent example of the use of followups. 
KEYWORDS: Followup: mail 
Incentive: postage. 


176. 
1926. The returns from followup letters to questionnaires. J. Appl. 
Psychol. 10(1): 92-101. 


Six followup letters resulted in 100-percent return from 110 questionnaires 
sent to college administrators asking about their use of intelligence tests. The 
relationship between use or nonuse of the tests and the number of followups 
required was examined to determine what caused the recipients to delay res- 
ponding to the questionnaire. Recipients who knew the sender replied sooner 
than strangers. All followup letters are illustrated. 

EVALUATION: Excellent example of effect of persistent followup. 
KEYWORDS: Followup: mail. 
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1935. Predicting the returns from questionnaires: a study in the utiliza- 
tion of qualitative data. J. Exp. Educ. 3(3): 204-215. 

Multiple regression analysis of two questionnaire investigations was used 
to determine which of 17 questionnaire characteristics were most closely related 
to maximum returns. Questionnaire factors which had the highest zero-order 
correlation with percentage of returns include questionnaire subject; number of 
copies issued; address of questionnaire sender; ease of answering questions; 
type of question asked (yes-no, short answer, or long answer); objective or 
subjective answers requested; and title of questionnaire sender. 

EVALUATION: Poorly written; conclusions weakened by presenting a summary 
of "seemingly important elements . . . which go beyond our data" and are 
plainly labeled as a ''guess.'' However, the factors can be used as pointers in 
questionnaire construction. 
KEYWORDS: Questionnaire: design 

Incentive: motivation. 


1937. The factor of mechanical arrangement and typography in question- 
naires. J. Appl. Psychol. 21(2): 225-229, 


The questionnaire should be viewed as a stimulus for achieving a particular 
end result, and in its construction other considerations must be subordinated to 
that purpose. Rules of rhetoric are important only inasmuch as they make 
questions more clear. Redundancy may be useful. [Illustrated is the use of 
certain capitalized words of a sentence so that an idea can be obtained at a 
glance of the capitalized portions. This facilitates both the respondent and the 
evaluator. 

EVALUATION: Theoretically sound idea, worth considering when constructing 
questions. 
KEYWORDS: Questionnaire: design 

Question: wording. 


Vernon, P. E. 
1939. Questionnaire, attitude tests and rating scales, p. 199-299. In 
F. C. Bartlett, The study of society. New York: The Macmillan Co. 


Sections of this chapter include advantages of the questionnaire, types of 
questionnaires, criticisms, precautions in applying questionnaires and inter- 
preting results, and the representativeness of results. 

EVALUATION: This is a broad overview; if more recent material is unavailable, 
this text is worth reading. 
KEYWORDS: Bias: representativeness 

Disadvantages of questionnaires 

Interpretation 

Textbook. 


Vincent, Clark E. 
1964. Socioeconomic status and familial variables in mail questionnaire 
responses. Am. J. Sociol. 69(6): 647-653. 


181. 


182. 


Data show that mail questionnaire research appears to elicit a dispropor- 
tionately high response from subjects whose self-reported backgrounds in "normal" 
lower-middle-class families skew familial data in the direction of the predom- 
inant textbook model of the nice, stable, middle-class family. Data were based 
on the returns from two California psychological inventory questionnaires sent 
to 517 senior class students in 1954 and again in 1959. A slight tendency was 
noted for the cooperative respondents to have a more "socially desirable, '' con- 
formist personality profile. When the distribution of subjects among three 
response groups (cooperative, post-card, and no-response) was examined, the 
cooperative group was most likely to have completed high school and be in cleri- 
cal and sales level jobs, whereas the no-response group more often had fathers 
of high educational levels and both high and low occupational levels. The data 
on intrafamily relationships provided the most impressive trend evidence of 
bias introduced by respondent-nonrespondent differences. The cooperative 
group consistently overrepresented positive intrafamily relationships and 
underrepresented negative relationships. Similarly, the respondent sample 
overrepresented subjects who in childhood were "verbally disciplined by mother" 
and underrepresented subjects who were “physically disciplined by father." 
EVALUATION: Relevant to mail questionnaire research in general with impli- 
cations for mail questionnaire research on the family in particular. 


KEYWORDS: Bias: nonresponse. 


Wagner, isabelle F. 
1939. Articulate and inarticulate replies to questionnaires. J. Appl. 
Psychol, 23(1): 104-115. 


In open-end questionnaires, some respondents are more articulate than 
others. To determine if this articulate minority is representative, 558 women 
were asked two open-end questions. Answers were rated for articulateness 
(defined as the total number of words used to reply). Results indicate (1) similar 
amounts of articulateness to both questions; (2) no relation to age, income, 
marital status, and husband's occupation; and (3) positive relation of habits and 
interests of articulate respondents to intellectual training and personality. 
EVALUATION: Assumption that articulateness equates to verbosity is question- 
able. Results are inconclusive. 

KEYWORDS: Open-end questions. 


Waisanen, F. B. 

1954. A note on the response to a mailed questionnaire. Publ. Opin. 

Quart. 18: 210-212. 

A sample of 300 TV and no-TV families were randomly selected in Iowa 
City. Phone calls were made to 175 of these families; of TV owners and nonowners 
who were phoned prior to the questionnaire mailing, 47.8 and 44.7 percent, 
respectively, returned the questionnaire within 10 days. The return from TV 
owners and nonowners who were not phoned was only 28 and 24.5 percent, re- 
spectively. Although limited to local research studies, the phone contact tech- 
nique provides personal contact, elicits the promise of cooperation, and pro- 
vides an opportunity to emphasize the brevity and simplicity of the questionnaire. 
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Phone contacts could have been made at the rate of 30 per hour, and of 175, only 
two refused to cooperate. 
EVALUATION: Good example of what personal attention to respondents can do 
for stimulating returns. With additional prompting and longer time period, 
greater returns could have been realized. 
KEYWORDS: Prequestionnaire letter , contact (phone) 

Bias: nonresponse. 


Wallace, David 
1954. A case for -- and against -- mail questionnaires. Publ. Opin. 
Quart. 18: 40-52. 


Two groups were compared: male 7ime magazine subscribers and the 
male population-at-large in Dayton, Ohio. Four questionnaires were sent 
successively to each group at intervals of 3 weeks. Approximately the same 
number of men replied to each mailing, and there was no marked variation in 
the response rates due to question content. There was little indication of re- 
spondent "'fatigue'' even after four questionnaires. In the "case for" the question- 
naire, the authors found that repliers and nonrepliers were almost the same 
over the four mailings. Typical mail surveys with 15- to 30-percent return rate 
often assume that the respondents are representative of the entire sample popu- 
lation. This is based upon the idea that for one reason or another some persons 
answered this time and those who did not will answer the next survey question- 
naire. In the ''case against'' mail questionnaires, the authors found that persons 
who answered any three or all four questionnaires is greater than the number 
who might be expected to do so by chance. Also, the number who did not 
answer any of the questionnaires was greater than chance expectations. In other 
words, typical mail response consists of upwards of 50 percent of habitual 
repliers and a disproportionately small number of persons who answer only 
occasionally. Thus, in surveys with low return rates, the response is rarely 
representative of the universe being sampled. 

EVALUATION: Excellent lesson on representativeness of low response rate 
surveys. One weakness is the apparent assumption that low return rates are 
inevitable with mail questionnaires; thus, the author is addressing himself to 
low return rate surveys and not mail surveys as such. 
KEYWORDS: Bias: nonresponse, representativeness. 


Wang, Charles K. A. 
1932. Suggested criteria for writing attitude statements. J. Soc. Psychol. 
3(3): 367-373. 

The statement must be debatable; all statements on a given issue should 
belong to the same attitude variable; statements should have only one interpre- 
tation; express only one idea; be short; use specific terms; avoid more than one 
sentence; not infer an attitude but be clear cut and direct; avoid words subject to 
interpretation such as only, mere, just, large, etc.; and avoid colorless expres- 
sions. Sentences should be simple rather than complex or compound. [If 
sentence cannot be simple, make it complex rather than compound; use the active 
voice; try to use the term of the issue as the subject of the sentence; and avoid 
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uncommon words, technical terms, negative expressions, double infinitives, 
redundant phrases, double negatives, and phrases such as "I think. . .,"' 
Mioebiengee Gua os 
EVALUATION: Excellent suggestions, highly recommended for anyone con- 
structing questions of any kind. 
KEYWORDS: Question: wording 

Questionnaire: design. 


Weilbacher, William M., and H. Robert Walsh 
1952. Mail questionnaires and the personalized letter of transmittal. 
J. Marketing 16(3): 331-336, illus. 


A total of 472 letters were mailed to university alumni fraternity members; 
46 were returned unclaimed, 184 returns were received, and 242 did not res- 
pond. The sample was divided so that half received a personalized letter of 
transmittal and a control group did not. There was no significant difference 
between the two groups. 
EVALUATION: Poorly written; findings may be questioned due to very low 
response rate. 
KEYWORDS: Cover letter 

Incentive: motivation. 


Wembridge, Eleanor Rowland, and Edger R. Means 
1918. Obscurities in voting upon measures due to double-negative. J. 
Appl. Psychol. 2: 156-163. 


Sixteen propositions were expressed in six different ways: simple affirma- 
tive, simple negative (with a word like prohibit), simple negative (with the word 
not), double negative (with words not prohibit), double negative (with prohibit 
prohibit), and double negative (with not not). Results show that test subjects 
took much greater time to answer the double negative questions than the simple 
affirmative ones. In addition, the double negative questions confused respon- 
dents as to the true meaning of the question. An example of the double negative 
is "minors shall not be forced not to smoke." 

EVALUATION: Excellent paper on a topic often taken for granted. 


KEYWORDS: Question: wording. 


Whitney, Frank P, 
1924. The questionnaire craze. Educ. Rev. 68(Oct.): 139-140. 
A barrage of questionnaires were sent to school administrators from 
irresponsible sources or merely curious parties. 
EVALUATION: An interesting commentary on the early misuse of questionnaires. 
KEYWORDS: Applications 
History. 


Wiiliams, Allan F., and Henry Wechsler 


1970. The mail survey: methods to minimize bias owing to incomplete 
response. Sociol. & Soc. Res. 54(4): 533-535. 


(fat 


A 20-item questionnaire was sent to 1, 695 dentists to determine willingness 
to expand their practice. Five contacts were made with respondents: prelim- 
inary letter, initial questionnaire, followup reminder, second questionnaire, 
and final followup reminder (each sent at 7-day intervals). Guidelines are pre- 
sented on presentation, content, and form of the questionnaire. Response rate 
was 77.8 percent. A telephone followup was conducted on the 440 nonrespondents 
1 month later. Results indicate that about 15 percent of the nonrespondents had 
moved and 20 percent agreed to return the questionnaire. Of the original 358, 

64 percent were reached on the first phone call and only 7.5 percent required 
more than three attempts. Respondents were typically younger specialists who 
worked fewer weeks per year than nonrespondents. When the telephone contact 
was treated as nonresponse and compared with earlier respondents, no statisti- 
cally significant differences were found. Telephone contacts are of value to 
determine eligible members of the sample and thus raise the response rate, 
and they also provide some information on nonrespondents. 
EVALUATION: Author presents an example of using Robin's system (see Robin 
1965) for minimizing bias in questionnaire research. 
KEYWORDS: Followup: mail, telephone 

Prequestionnaire letter 

Questionnaire: design. 


189. Wilson, P. W. 
1934. A question of questionnaires. N. Am. Rev. 237(4): 325-330. 


Essay criticizing questionnaires because they invade the privacy of the 
individual and can be used to ''manage our minds for us.'' Questionnaires not 
only yield answers but affect the respondent as well. If psychologists ask 
questions for the sake of their science, we may expect politicians to ask ques- 
tions for the sake of their politics. 

EVALUATION: Opinions poorly supported. 
KEYWORDS: Disadvantages of questionnaires. 


190. Young, Pauline V. 
1939. Schedule and the questionnaire as aids in field exploration, p. 138- 
171. In Scientific social surveys and research. New York: 
Prentice Hall. 

Instructions for the construction of questionnaires are presented, including 
suitable topics for maximum utility of the questionnaire, times for use, and 
suggestions for reducing nonresponse. Gives 11 checks used to evaluate the 
completed questionnaire. 

EVALUATION: Practical; organized to facilitate comprehension of essential 


points. 

KEYWORDS: Questionnaire: design 
Applications 
Textbook. 


ilgyale 
1956. The questionnaire and other reporting forms as aids in field 
exploration, chapter VIII, p. 176-204. In Scientific social 
surveys and research. Englewood Cliffs, N.J.: Prentice Hall. 
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192. 


193. 


Types of questionnaires are discussed along with aspects of design, com- 
munication problems, and bias. Under questionnaire format, the author dis- 
cusses question arrangement, type used in printing, kind of paper used, margins, 
spacing, cover letter and followup letters. 

EVALUATION: Practical and organized to facilitate comprehension of essential 


points. 

KEYWORDS: Questionnaire: design 
Applications 
Textbook. 


Young, R. A., I. I. Holland, and A. R. Gilmore 
1970. Getting better returns from mail questionnaires. J. For. 68: 
723-724, illus. 


A mail questionnaire case study of 500 randomly selected Illinois campers 
showed an 85-percent return. The high rate of return was attributed to the 
questionnaire's appearance, simple layout, and wording; a pretest; a personalized 
cover letter; stamped return envelope; and two followup letters. 

EVALUATION: Good practical questionnaire procedures outlined. Article fails 
to point out importance of sampling nonrespondents to determine if they differ 
from respondents. 
KEYWORDS: Questionnaire: design 

Cover letter 

Incentive: postage 

Followup: mail. 


Zimmer, Herbert 
1956. Validity of extrapolating nonresponse bias from mail questionnaire 
follow-ups. J. Appl. Psychol. 40(2): 117-121. 

A study tests the hypothesis that response-nonresponse probability function 
indicates the presence and direction of nonresponse bias. Questionnaires were 
mailed to 220 servicemen; and after one followup and 11 months, 87 percent had 
been returned. Results indicate that five (age, education, rank, years in service, 
and occupation) of the seven variables considered were found to be consistent 
with the hypothesis but failed to attain statistical significance. Marital and 
military status each contained significant nonresponse bias but was inconsistent 
with the hypothesis. 

EVALUATION: Excellent attempt to predict nonresponse bias. Failure to 
prove hypothesis may be due to small sample size. 
KEYWORDS: Bias: nonresponse 

Validity. 
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The mission of the PACIFIC NORTHWEST FOREST 
AND RANGE EXPERIMENT STATION is to provide the 
knowledge, technology, and alternatives for present and 
future protection, management, and use of forest, range, and 
related environments. 


Within this overall mission, the Station conducts and 
stimulates research to facilitate and to accelerate progress 
toward the following goals: 


1. Providing safe and efficient technology for inventory, 
protection, and use of resources. 


2. Development and evaluation of alternative methods 
and levels of resource management. 


3. Achievement of optimum sustained resource produc- 
tivity consistent with maintaining a high quality forest 
environment. 


The area of research encompasses Oregon, Washington, 
Alaska, and, in some cases, California, Hawati, the Western 
States, and the Nation. Results of the research will be made 
available promptly. Project headquarters are at: 


Fairbanks, Alaska Portland, Oregon 
Juneau, Alaska Olympia, Washington 
Bend, Oregon Seattle, Washington 
Corvallis, Oregon Wenatchee, Washington 


La Grande, Oregon 


GPO 796-430 
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